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PREFACE 


This  project  was  initiated  as  a  result  of  a  concern  that  had  been  nagging  at  me  since  assuming 
leadership  of  the  division  in  April  1991 — almost  four  years  prior  to  the  start  of  this  effort.  At  that 
time,  it  was  evident  that  though  the  division  had  responsibility  over  a  well-defined  scope  of  human 
factors  R&D,  our  resident  expertise  in  most  areas  was  typically  but  one  deep.  Moreover,  we  were 
locked  into  a  hiring  freeze  while  individual  experts  dropped  out  in  retirement  and  the  organization 
grew  progressively  grayer.  This  irreplaceable  loss  of  corporate  expertise  was  challenging  in  several 
ways. 

Foremost  was  our  concern  for  sustaining  a  corporate  presence  in  key  product  areas  where  the 
Human  Engineering  Division  had  an  historic  presence  based  on  significant  contributions.  Of 
almost  equal  concern  was  a  gnawing  fear  and  realization  that  this  loss  of  key  personnel  irrevocably 
diminished  the  success-oriented  intellectual  capital  and  skilled  critical  mass  vital  to  the  operation  of 
the  unit. 

As  key  individuals  retired  and  departed  from  the  division,  few  traces  beyond  formal  professional 
communications  remained  to  provide  insight  as  to  the  factors  which  motivated  them  to  success  and 
the  nature  of  the  lessons  they  learned  by  experience  which  might  be  offered  to  future  generations  of 
human  engineers  following  in  their  footsteps. 

Fearing  this  loss  of  experience,  I  contacted  Gary  Klein  who  had  previously  conducted  an 
innovative  study  at  the  Wright  Laboratory  at  Wright-Patterson  Ara  to  identify  the  contributing 
factors  to  the  success  of  a  distinguished  group  of  managers,  project  engineers  and  S&Es.  Intrigued 
by  the  product  of  this  effort,  I  engaged  Gary  in  a  discussion  as  to  how  his  "case-based"  interview 
techniques  could  similarly  be  put  to  work  to  extract  the  underlying  critical  variables,  heuristics  for 
success,  and  lessons  learned  from  the  experience  base  of  a  selection  of  senior  and  retired  S&Es 
from  the  Armstrong  Laboratory  Human  Engineering  Division. 

This  report  documents  selected  high  points  from  their  successful  careers  and  attempts  to  extract 
themes,  attributes,  and  variables  common  to  the  case  studies  that  are  cited.  Our  hope  is  that  this 
information  can  be  useful  as  guidance  to  individual  scientists,  while  also  influencing  the  evolving 
processes,  plans  and  operations  of  the  Human  Engineering  Division. 


Kenneth  R.  Boff,  Ph.D. 

Chief,  Human  Engineering  Division 


May  1995 


RCKNOUILEDGEMENTS 


This  effort  was  initiated  by  Ken  Boff,  who  wanted  to  find  a  different  way  to  tell  the  story  of  the 
Human  Engineering  Division  in  addition  to  bibliographies  and  historical  reviews.  We  are  grateful 
for  his  assistance  in  shaping  the  project  and  carrying  it  through.  We  also  want  to  express  our 
thanks  to  Lew  Hann,  Tanya  Ellifritt,  and  Betsy  Combs  at  Human  Engineering  Division  for  their 
roles  in  helping  us  move  the  project  along. 

We  are  also  appreciative  of  the  help  given  by  the  researchers  we  interviewed:  Earl  Sharp,  Phil 
Kulwicki,  Charlie  Clauser,  Don  Topmiller,  Dave  Post,  Bob  O’Donnell,  Mel  Warrick,  Julien 
Christensen,  Steve  Heckart,  Jerry  Chubb,  and  Lee  Task.  They  were  generous  with  their  time  in 
sitting  through  a  two  hour  interview  (and  in  some  cases  more  than  one  interview),  and  in 
reviewing  the  written  record  to  correct  inaccuracies.  More  important,  they  were  generous  with  their 
memories  and  impressions  as  they  wrestled  to  figure  out  just  how  they  had  solved  those  problems. 

We  also  want  to  express  our  gratitude  to  our  colleagues  at  Klein  Associates:  Buzz  Reed,  Beth 
Crandall,  Caroline  Zsambok,  Barbara  Law,  Mary  Alexander,  Debbie  Goessl,  and  Diane  Iglesias 
for  their  assistance  in  producing  this  document,  including  reviewing  and  editing  the  materials  and 
making  helpful  suggestions  for  improvement. 

The  project  was  supported  by  contracts  and  funding  from  University  of  Dayton  Research 
Institute,UDRI-82252X.,  and  Logicon  Technical  Services  Inc. 


iv 


The  Innouators:  High-Impact  Researchers 
at  the  Armstrong  Laboratory  Human 
Engineering  Diuision 

TABLE  OF  CONTENTS 

INTRODUCTION . 1 

CASE  STUDIES . 3 

Honey,  I  Shrunk  the  Color  TV:  The  Case  of  the  Misleading  Metaphor . 4 

Dave  Post 

The  Bomber  Simulation,  Test,  and  Evaluation  Facility:  The  Million  Dollar  Meeting . 8 

Earl  Sharp 

The  Case  of  the  Fat  Cursors:  The  B-2  Synthetic  Aperture  Radar .  12 

Earl  Sharp 

Flying  Ahead  of  the  High-G  Cockpit . 12 

Phil  Kulwicki 

Targets  of  Opportunity:  The  Case  of  the  Corrupted  Cues .  22 

Phil  Kulwicki 

On  Top  of  the  World:  The  Case  of  the  Confusing  Compasses .  26 

Julien  Christensen 

Going  Ballistic  for  Human  Factors  Standards . . . 30 

Donald  Topmiller 

The  Case  of  the  Uncommon  Consoles .  33 

Steve  Heckart 

“Measure  Me  Workload!” .  35 

Bob  ODonnell 

Moments  of  Inertia .  38 

Charlie  Clauser 

Crossing  the  Barrier:  Developing  SAINT  to  Model  Human  and  Equipment  Dynamics .  41 

Jerry  Chubb 

The  Case  of  the  Wavy  Windscreens .  45 

H.  Lee  Task 


The  Pilots  who  had  to  be  Kept  in  the  Dark . 47 

H.  Lee  Task 

The  Pilots  who  had  to  Play  it  by  Ear .  49 

H.  Lee  Task 

Warrick's  Principle:  The  Case  of  the  Compatible  Controls . 51 

Mel  Warrick 

DISCUSSION .  53 


Vi 


The  Innouators:  High-Impact  Researchers 
at  the  Human  Engineering  Diuision, 
Hrmstrong  Laboratorg 

Authors:  Gary  Klein  and  Rob  Hutton,  Klein  Associates  Inc. 

INTRODUCTION 


This  report  describes  a  set  of  successful  research  programs  at  the  Armstrong  Laboratory  Human 
Engineering  Division.  The  goal  of  the  effort  was  to  conduct  in-depth  interviews  with  a  very  small 
sample  of  behavioral  scientists  and  engineers  at  the  Human  Engineering  Division,  in  order  to 
formulate  some  hypotheses  about  how  a  researcher  comes  to  have  a  strong  impact  on  user 
communities  in  the  Air  Force. 

For  each  of  the  interviews,  we  asked  the  researcher  to  discuss  a  project  that  had  a  clear  benefit.  “If 
the  project  was  a  story,”  we  began,  “the  end  of  the  story  would  be  that  the  sponsors  or  users  were 
enormously  grateful  and  immediately  benefitted  from  the  findings  or  product.  Give  us  an  example 
that  has  this  type  of  ending.”  Some  of  the  interviewees  came  up  with  one  or  even  several  examples; 
others  struggled  because  there  was  no  clearly  defined  sponsor,  even  though  the  findings  were 
valuable  to  a  larger  community.  The  interviews  were  therefore  both  challenging  and  engaging  for 
the  interviewers  as  well  as  the  researchers. 

Each  researcher  was  interviewed  individually.  The  interviews  typically  lasted  for  two  hours.  The 
researcher  described  a  high-impact  project,  and  we  attempted  to  learn  more  about  the  decisions 
within  the  project,  the  types  of  critical  judgments,  and  the  problem-solving  activities  involved.  We 
were  trying  to  identify  and  understand  a  number  of  the  characteristics  of  high-impact  programs: 

How  were  opportunities  detected? 

To  what  extent  did  the  success  depend  on  individual  initiative? 

What  was  the  time  course  of  the  project,  and  of  the  benefits? 

How  were  the  researchers  carrying  out  the  strategic  goals  of  the  Division? 

How  did  the  researchers  work  through  bureaucratic  barriers? 

What  were  the  team  dynamics  of  the  effort? 

Our  objective  was  to  identify  common  themes.  In  so  doing,  we  were  hoping  to  assist  the  Human 
Engineering  Division  in  replicating  these  success  stories.  There  are  several  ways  that  the  Division 
can  use  these  case  studies.  One  way  is  to  compile  the  lessons  learned  and  begin  to  build  a  corporate 
memory  to  enable  organizational  learning  to  go  on.  A  second  way  is  to  better  understand  the  type 
of  organizational  support  needed  to  increase  the  success  rate.  A  third  way  is  to  use  these  case 
studies  as  models  so  that  new  scientists  and  engineers  will  have  a  better  idea  of  what  is  expected  of 
them.  A  fourth  way  is  to  consider  organizational  design  issues  that  might  increase  impact. 

There  was  another  objective  guiding  this  effort:  to  present  a  picture  of  the  way  researchers  at  the 
Armstrong  Laboratory  Human  Engineering  Division  do  their  jobs.  Technical  reports  do  not  always 
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convey  the  way  the  research  was  conducted — the  frustrations,  confusions,  and  satisfactions.  In 
preparing  the  case  studies  we  were  seeking  to  provide  a  different  perspective  on  how  the  Human 
Engineering  Division  works  to  support  the  goals  and  activities  of  the  U.S.  Air  Force. 

The  case  studies  described  in  this  report  are  not  intended  as  a  review  of  the  work  of  the  Human 
Engineering  Division.  Only  fifteen  projects  are  described,  based  on  interviews  with  only  eleven 
researchers.  Clearly,  the  great  majority  of  effective  researchers  and  recognized  accomplishments 
are  not  covered.  The  selection  of  people  and  projects  placed  an  emphasis  on  senior  researchers  to 
try  to  take  advantage  of  their  perspective  on  the  Division’s  activity.  The  majority  of  those 
interviewed  are  currently  retired.  The  case  studies  cut  across  decades.  The  earliest  are  from  the 
very  beginning  of  the  Human  Engineering  Division  in  the  late  1940s,  and  the  most  recent  are 
efforts  that  are  still  underway.  Many  changes  have  occurred  during  these  fifty  years.  We  have  tried 
to  identify  factors  that  are  time-independent,  to  show  the  working  styles  of  researchers  who  have 
been  able  to  make  a  difference. 

The  written  accounts  of  the  case  studies  are  intended  to  convey  the  story  line  of  the  project, 
highlighting  some  of  the  themes.  The  reader  may  derive  different  conclusions  than  the  ones  we 
obtained,  and  we  hope  that  our  project  descriptions  are  sufficiently  clear  to  allow  this  type  of 
constmctive  reading.  Readers  who  seek  additional  information  about  individual  projects  are 
encouraged  to  contact  the  researchers  themselves,  or  to  review  the  relevant  technical  reports  listed 
at  the  end  of  some  of  the  stories. 
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CHSE  STUDIES 


Honey,  I  Shrunk  the  Color  TU:  The  Case  of 
the  Misleading  Metaphor 

An  Interview  with  Dave  Post 


Sometimes  we  can  become  trapped  by  a  metaphor.  The  idea  of  a  miniaturized  color  display  carries 
a  compelling  metaphor:  a  conventional  color  TV  that  just  has  to  be  reduced  in  size.  But  this 
metaphor  contains  a  fatal  flaw.  The  development  of  the  Miniaturized  Color  Display  depended  on 
Dave  Post’s  rejection  of  the  dominant  metaphor. 

Post  received  his  Ph.D.  in  Human  Factors  from  Virginia  Polytechnical  Institute  and  started  work  at 
Armstrong  Laboratory  in  1983,  in  the  Visual  Display  Systems  Branch,  working  in  the  area  of  color 
research.  Tom  Furness  was  the  branch  chief.  Post  took  the  job  because  he  liked  the  idea  of  helping 
to  form  his  own  programs;  the  jobs  he  had  considered  in  industry  paid  more,  but  were  restricted  as 
to  what  topics  could  be  investigated.  Upon  arrival  at  the  laboratory,  he  enjoyed  the  freedom.  He 
worked  in  the  areas  of  helmet-mounted  displays,  and  also  performed  some  studies  on  the  legibility 
of  color  symbology.  After  Post  had  spent  a  few  years  in  the  branch,  Tom  Furness  asked  him  to  try 
to  build  a  color  version  of  a  miniature  CRT,  and  he  found  it  an  interesting  challenge,  but  the  project 
ran  into  practical  difficulties  and  was  cancelled.  Nevertheless,  it  did  get  Dave  interested  in  the 
question  of  how  to  put  color  into  helmet-mounted  displays. 

The  Tradeoff 

Color  televisions  come  in  all  sizes,  from  giant  screens  to  portables.  However,  there  is  a  practical 
limit  to  how  small  they  can  be.  The  limit  is  based  on  resolution.  When  we  reduce  the  size  of  a 
black-and-white  CRT,  the  resolution  suffers.  But  resolution  degrades  much  more  severely  with 
color.  The  reason  is  that  CRTs  rely  on  three  primary  colors,  red,  green  and  blue  dots,  to  form 
other  colors.  Each  color  is  represented  by  different  proportions  of  the  three  primary  colors,  and 
these  fuse  at  a  distance,  appearing  to  be  whatever  color  is  being  mixed.  This  works  very  well  on  a 
TV,  with  a  big  screen,  seen  at  a  far  distance.  It  is  not  OK  for  a  helmet-mounted  display.  The  use  of 
three  primary  colors  translates  into  a  3:1  loss  of  resolution.  It  takes,  on  the  average,  three  times  as 
many  dots  on  a  color  CRT  to  create  the  same  picture  on  a  black-and-white  screen.  When  you 
reduce  the  size  of  the  CRT  to  a  few  inches,  the  lost  resolution  makes  the  image  too  fuzzy  to  see. 

As  a  result,  designers  have  to  trade  off  color  for  resolution.  The  3: 1  ratio  means  that  a  display  has 
to  be  at  least  three  times  as  large  in  a  color  version  as  in  a  black-and-white  version,  and  even  then 
the  primary  colors  might  not  fuse  adequately  if  the  display  is  too  close  to  the  eye.  This  would  seem 
to  rule  out  the  possibility  of  color  versions  of  helmet-mounted  displays. 

Actually,  for  helmet-mounted  displays,  the  problem  is  even  more  serious  because  these  displays 
have  to  be  useable  even  under  bright  daylight.  The  displays  need  lots  of  luminance,  since  the  pilots 
have  to  be  able  to  see  the  symbology  against  a  bright  sky.  The  increased  luminance  of  the  sky 
reduces  the  display  contrast  even  more,  making  the  resolution  worse  yet. 

A  Funding  Vehicle  Opportunity 

Around  1987  Tom  Furness  began  to  think  about  using  a  new  contracting  vehicle,  the  Program 
Research  and  Development  Announcement  (PRD A)  to  fund  some  work  to  look  at  putting  color  in 
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helmet-mounted  displays.  The  PRDA  was  non-traditional  and  was  designed  to  encourage  creativity 
and  innovation.  The  idea  is  to  state  a  problem  and  let  vendors  suggest  the  program  needed.  The 
traditional  approach  is  to  define  the  program  and  have  contractors  bid  on  it.  Furness  wanted  to  see 
how  the  PRDA  would  work,  and  he  thought  that  the  challenge  of  putting  color  into  helmet- 
mounted  displays  was  a  good  place  to  start.  Under  the  PRDA  concept,  proposals  were  evaluated 
on  the  basis  of  the  innovative  ideas.  Multiple  awards  were  possible.  The  plan  Tom  and  Dave  drew 
up  was  to  make  three  $600K  awards  for  a  two-year  project.  The  first  phase  was  just  to 
demonstrate  the  feasibility  of  the  technologies  proposed.  These  would  be  evaluated  to  see  which,  if 
any,  to  continue  to  a  second  phase. 

Seeing  the  Light 

In  preparing  the  PRDA  package.  Post  reviewed  the  state  of  the  art.  What  he  found  did  not  offer 
much  encouragement.  He  confirmed  his  suspicion  that  the  shadowmask  design  used  in  color 
televisions  would  not  work  because  of  the  3: 1  resolution  loss.  Projection  systems  were  too  bulky 
to  be  used  in  cockpits.  A  field  sequential,  additive  strategy  threw  away  too  much  light,  and 
suffered  problems  in  which  the  color  broke  up  (i.e.,  the  primary  colors  didn't  fuse  in  a  satisfactory 
way).  Post  looked  at  lasers,  but  these  suffered  from  poor  efficiency.  Lasers  also  had  another 
difficulty — the  laser  images  had  no  persistence,  unlike  phosphors.  That  meant  that  the  images  were 
going  to  be  choppy  and  disorienting  to  the  pilots. 

While  looking  through  the  literature  on  color  images.  Post  reread  a  book  by  Bob  Hunt  on 
photography.  Hunt  had  worked  for  Kodak  for  most  of  his  career,  and  had  prepared  a  highly 
regarded  work.  Going  through  the  chapter  on  color  photographs.  Post  thought  about  the  fact  that 
color  photographs  are  much  sharper  than  color  CRT  images.  The  reason  is  that  color  photographs 
and  slides  don’t  use  an  additive  method  of  combining  red,  green  and  blue.  Instead,  they  rely  on  a 
subtraction  method.  They  consist  of  three  layers.  Each  layer  filters  out  light  at  a  different 
wavelength.  The  layers  correspond  to  magenta,  cyan,  and  yellow  as  the  primary  colors  for 
subtraction.  White  light  is  filtered  to  remove  portions  of  the  spectrum.  This  works  extremely  well. 
The  resolution  of  a  color  photograph  or  slide  is  about  the  same  as  a  black-and-white  image. 

Post  wondered  if  this  color  subtraction  approach  could  be  used  with  transmissive  displays.  He 
imagined  how  you  could  stack  three  Liquid  Crystal  Displays  on  top  of  each  other  to  do  the 
subtraction.  If  this  could  be  done,  then  he  wouldn't  have  to  be  trapped  by  the 
color/resolution  tradeoff.  He  could  get  both  good  color  and  high  resolution.  Post  knew  that 
the  technology  wasn't  out  there  to  make  this  work,  and  he  worried  that  the  multiple  filtering  would 
cut  luminance  too  much,  but  he  tucked  the  idea  away.  It  gave  him  confidence  that  there  was  at  least 
one  path  worth  exploring. 

Soon  thereafter.  Post  attended  a  meeting  of  the  Society  for  Information  Display.  He  talked  to 
different  vendors  selling  various  types  of  displays,  to  try  to  dmm  up  interest  in  the  PRDA  he  was 
preparing.  But  he  got  the  same  response,  “Well,  I  just  don’t  see  how  you’re  going  to  miniaturize  a 
shadowmask  TV  down  to  one  inch.” 


The  idea  bf  miniaturizing  a  color  TV  was  such  a  powerful  metaphor  for  a  miniaturized  color 
display  that  the  vendors  were  trapped  into  the  color  addition  strategy  used  by  televisions. 
Their  thinking  was  restricted  by  the  metaphor. 
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Towards  the  end  of  the  conference.  Post  ran  into  an  old  friend  who  listened  to  the  description  of 
the  PRDA  and  introduced  him  to  an  engineer  working  for  Sperry  (in  a  division  that  later  was 
acquired  by  Honeywell).  Over  lunch,  the  engineer  described  an  earlier  effort  which  carried  out  a 
color  subtraction  strategy.  The  engineer  had  set  up  a  stack  of  LCDs,  built  a  prototype,  and  patented 
it.  The  prototype  was  around  four  to  five  inches,  and  required  direct  viewing. 

Post  recognized  that  this  was  a  great  idea,  especially  since  it  matched  the  idea  for  color  subtraction 
he  already  had  formed.  He  had  not  wanted  to  impose  this  color  subtraction  concept  on  any  vendors 
since  it  would  defeat  the  purpose  of  the  PRDA,  to  encourage  a  variety  of  approaches. 

Nevertheless,  Post’s  confidence  in  the  project  went  up  now  that  he  knew  that  the  color  subtraction 
method  was  more  than  a  curious  lead.  He  returned  to  Wright-Patterson  and  got  the  PRDA  released. 
Many  vendors  submitted  proposals,  including  Sperry/Honeywell. 

Then  everything  shut  down.  The  money  for  the  PRDA  was  supposed  to  come  through  the 
Armstrong  Laboratory’s  Supercockpit  program,  but  that  program  disappeared.  There  was  no 
funding  for  any  of  the  proposals. 

Running  on  Empty 

That  should  have  been  the  end  of  the  project,  since  the  rationale  had  been  to  explore  the  PRDA 
mechanism  as  a  strategy  for  working  with  industry.  The  rationale  was  gone.  And  so  was  the 
funding.  It  was  time  to  give  up  and  shift  to  other  projects. 

But  by  this  time,  Post  had  become  committed  to  the  color  subtraction  strategy.  He  kept  looking  for 
alternative  funding,  and  in  1989  a  colleague  in  the  branch.  Dean  Kocian,  came  up  with  $600,000. 
This  would  be  only  enough  to  fund  a  single  effort,  but  that  was  enough.  Post  went  through  the 
process  and  reissued  the  PRDA  in  the  Commerce  Business  Daily.  He  received  around  17 
proposals,  which  was  more  than  he  expected.  He  filled  the  tmnk  of  his  car  with  proposals  on  a 
Friday  afternoon,  and  spent  all  weekend  reading  them.  Honeywell  submitted  the  only  proposal 
based  on  color  subtraction.  Post  satisfied  himself  that  none  of  the  other  approaches  was  as 
promising,  and  by  Monday  morning  he  knew  who  should  receive  the  contract.  The  rest  of  the 
procurement  process  proceeded,  including  gathering  together  an  evaluation  team  and  conducting 
formal  assessment,  a  process  that  dragged  out  for  about  six  months. 

During  1991-1993  he  monitored  the  contract.  The  prototype  was  as  successful  as  expected.  With 
the  stacked  LCDs,  each  pixel  was  a  full-color  pixel  so  there  was  no  resolution  penalty.  Some  team 
members  worried  that  after  the  light  passed  through  the  filters,  the  display  would  be  dim.  This  was 
not  the  case.  To  increase  luminance,  you  just  need  a  more  powerful  light  source.  Furthermore,  the 
component  on  the  helmet  doesn't  have  to  generate  light  (since  you  can  use  fiber  optics  and  have  a 
remote  lamp),  so  there  is  no  weight  penalty.  There  were  some  concerns  that  the  device  would  be 
too  hot,  changing  the  properties  of  the  LCDs,  but  this  proved  not  to  be  a  problem.  Everything  went 
well.  Except  for  one  little  difficulty.  When  the  project  was  finished,  there  was  no  money  for  a 
Phase  n,  to  take  the  project  any  further.  Again,  the  project  had  mn  into  a  roadblock. 

Fortunately,  ARPA  was  investigating  High  Definition  Television  (HDTV)  at  the  same  time.  One  of 
the  gaps  in  the  ARPA  program  was  small,  helmet-mounted  displays.  Dick  Urban  at  ARPA  took  on 
this  task,  and  Henry  Girolamo  of  the  Army  suggested  that  they  set  up  a  Tri-Service  Advisory 
Group  to  guide  the  effort.  This  included  NASA  along  with  the  three  military  services.  The  Tri- 
Service  Group  held  several  meetings,  and  included  Bob  Michaels  and  some  other  engineers  from 
Wright  Laboratories.  Michaels  already  knew  about  Post’s  project,  and  feeling  outnumbered  by  the 
other  services,  Michaels  contacted  Dave  Post  and  invited  him  to  attend  the  Tri-Service  meetings. 
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Post  accepted  the  invitation,  and  discovered  that  the  group  was  considering  conventional  color 
filter  LCDs  and  active  matrix  electroluminescent  technologies.  This  was  all  well  and  good  for 
ground-based  displays  but  did  not  provide  the  required  luminance  contrast  for  airborne  operations. 
Bob  Michaels  and  Dave  Post  quicldy  convinced  the  group  that  they  needed  to  rely  on  color 
subtraction  methods. 

In  short  order,  the  Tri-Service  Group  funded  the  Phase  n  project  to  follow  on  the  work  Post  had 
been  monitoring.  Even  better.  Post  no  longer  had  the  responsibility  to  carry  out  the  procurement 
tasks  of  getting  the  contract  awarded.  The  contract  went  to  a  team  that  included  Standish  (to 
assemble  the  LCDs),  Honeywell  (to  assemble  the  subtractive  stacks),  David  Samoff  Laboratories 
(to  design  the  circuit),  and  Kopin  (to  build  the  miniaturized  circuitry).  Post  was  the  technical 
contract  monitor.  The  Phase  n  carried  out  the  original  ideas  and  pushed  them  further. 

Currently,  the  display  has  an  active  area  of  1.3  inches  by  one  inch.  The  resolution  is  640x480 
pixels,  and  a  1280x1024  pixel  version  is  in  the  works.  The  miniaturized  color  display  has  the 
resolution  and  the  luminance  it  needs.  The  Army  has  recently  used  the  subtractive  technolo^ 
system  for  a  color  binocular  helmet.  The  Air  Force  may  use  the  miniaturized  color  displays  in  a 
follow  on  to  its  monochrome  VCATS  (Visually  Coupled  Acquisition  and  Targeting  System) 
device.  Additional  applications  of  light-weight  miniature  color  displays  are  being  considered  in 
medical  endoscopy,  entertainment  (including  HDTVs  of  the  future)  and  virtual  reality. 
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The  Bomber  Simulation,  Test,  and 
Eualuatlon  Facility:  The  Million  Dollar 

Meeting 

An  Interview  with  Earl  Sharp 


The  Armstrong  Laboratory  at  Wright-Patterson  AFB  has  proudly  been  the  owner  of  some  of 
the  most  realistic  simulation,  test,  and  evaluation  facilities  for  bomber  crew  stations  in  the 
world,  thanks  in  large  part  to  the  early  efforts  of  Earl  Sharp. 

Earl  has  just  retired,  but  his  legacy  is  the  B-2  test  and  evaluation  facility,  the  latest  in  the  line  of 
bomber  simulators  with  sophisticated  data  collection  capabilities  for  human  factors 
research.  The  first  of  these  state-of-the-art  facilities  was  the  B-52  facility  which  Earl  set  up  in 
1971. 

Earl  is  a  well-known  character  among  the  bomber  crews,  SPOs,  and  SAC.  His  research 
program  for  the  defensive  crew  stations  of  the  B-52,  B-1,  and  B-2  are  known  to  provide 
realistic  mission  scenarios  for  the  purpose  of  testing  and  evaluating  new  concepts  in  crew 
station  design,  from  the  physical  layout  of  the  work  station  itself  to  such  details  as  the  design 
of  the  Synthetic  Aperture  Radar  (SAR)  target  designation  cursor  (see.  The  Case  of  the  Fat  Cursors: 
The  B-2  Synthetic  Aperture  Radar  -  An  Interview  with  Earl  Sharp). 


This  firslfaCilii^  came  abdut  through  a  series  of  aepidents,  such  as  a  casual  visit  to  a  friend  on 
base,  a  curiosity  about  some  videotapes,  and  a  chance  meeting  in  an  airline  terminal.  The  final 
accident  was  when  Earl  decided  to  ask  for  $3500  in  funding  and  walked  away  with  more  than 
$1M  in  suppoit. 


■Tust  Visiting  a  Friend  on  Base 

Earl’s  interest  in  the  Electronics  Warfare  Officer’s  (EWO)  workstation  was  sparked  by  chance  in 
1969.  He  had  left  his  office  with  the  typical  comment  “I’m  just  going  to  see  a  friend  on  base.” 
When  he  got  to  his  friend’s  laboratory,  he  realized  he  had  come  at  a  bad  time,  since  his  friend  was 
still  busy  analyzing  some  videotapes  of  the  B-52  EWO  at  work.  Instead  of  leaving,  Earl  stayed  to 
look  at  the  tapes.  Earl  had  heard  about  electronic  counter-measures  from  his  branch  chief  and  was 
curious  to  see  how  the  EWO  actually  did  his  job.  He  found  the  videotapes  fascinating,  watching 
the  EWO  stretching  and  reaching  and  jumping  up  to  see  hard-to-read  instruments. 

Earl  asked  if  he  could  help  with  the  videotape  data  analysis  and  became  engrossed  in  the 
possibilities  for  human  factors  work  on  this  crew  station.  He  began  to  attend  B-52  task  force 
meetings  addressing  human  factors  issues  in  bomber  aircraft. 
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And  For  Mv  Next  Wish 


After  sitting  in  as  an  observer  in  many  task  force  meetings,  trying  to  get  an  understanding  of  the 
human  factors  issues  involved  in  the  bomber  crew  stations,  Earl  had  gotten  to  know  the  people  and 
the  issues.  He  had  started  toying  with  the  idea  of  setting  up  a  simulation,  test,  and  evaluation 
facility  to  study  the  human  factors  issues  in  the  EWO  workstation  and  perhaps  the  others  as  well. 

In  1971  Earl  was  invited  to  attend  an  Air  Staff  B-52  task  force  meeting  at  Oklahoma  City  ALC.  He 
was  waiting  in  the  PAX  terminal  on  his  way  to  Oklahoma  City  and  got  to  talking  with  the  head  of 
the  task  force  about  his  ideas  for  a  full  mission  simulation  and  evaluation  device  for  human  factors 
research.  Earl  was  asked  if  he  would  like  to  present  his  ideas  to  the  task  force.  Earl  agreed,  since 
this  was  a  good  chance  to  put  his  ideas  into  action. 

He  realized  he  would  be  asked  for  a  budget  estimate  to  go  along  with  the  program  he  was 
suggesting,  and  he  didn’t  want  to  request  too  much  for  fear  of  looking  greedy  or  making 
unrealistic  demands.  He  also  knew  he  was  good  at  getting  makeshift  equipment  made,  so  he 
decided  that  he  could  do  the  work  for  $3500.  That’s  what  he  would  ask  for. 

However,  in  talking  to  some  friends  just  before  the  meeting  started,  he  got  some  feedback  that  his 
$3500  estimate  was  unrealistically  low.  They  told  him  it  would  cost  at  least  a  million  dollars  for  the 
facility  he  would  need.  After  juggling  some  numbers  with  a  few  people,  and  working  up  his 
courage,  he  came  up  with  a  figure  of  $270,000,  which  he  put  to  the  committee. 

The  task  force  was  ready  for  his  ideas.  He  walked  out  of  the  meeting  with  $270,000  in  his  pocket, 
and  swears  he  could  have  told  them  the  weather  and  the  time  of  day  and  all  they  would  have  asked 
was,  "When  do  you  want  the  money?"  Not  only  that,  but  he  persuaded  SAC  to  give  him  an  old  B- 
52  simulator  on  which  he  could  base  his  facility,  and  two  top-of-the-line  simulator  technicians  to 
help  him  set  it  all  up.  All  told,  the  value  of  the  package  was  more  than  a  million  dollars,  including 
the  simulation  facility,  the  technicians,  and  the  funds. 

In  six  months  the  facility  was  up  and  running  with  a  little  spare  money  to  run  some  initial 
studies.  The  facility  consisted  of  the  simulator  itself  plus  the  capability  of  collecting 
performance  data  about  every  flipped  switch,  turned  dial,  and  moved  cursor.  Earl  could  see 
what  was  pushed,  when  it  was  pushed,  what  it's  starting  position  was,  what  it's  end  position 
was,  how  long  it  took  to  push  it,  and  what  the  resulting  input  did.  The  crews  were 
videotaped,  as  well  as  observed.  They  were  debriefed  using  the  videotape  to  try  to  understand 
what  they  were  doing,  and  why  they  were  doing  it.  The  facility  provided  the  capability  to 
physically  modify  the  workstation,  as  well  as  change  the  input  devices  and  displays. 


Earl  had  been  a  machinist  before  he  had  begun  to  get  interested  in  human  factors,  so  when  it 
came  time  to  start  rebuilding  and  modifying  the  simulators  to  suit  his  needs  he  was  able  to  go 
down  to  the  local  machine  shop  on  base  and  tell  the  machinists  exactly  what  he  needed  in  their 
language.  In  this  way  he  was  able  to  get  the  facility  set  up  extremely  quick. 
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Putting  the  Simulation  to  Work 

One  of  Earl's  first  projects  was  to  redesign  the  workstation  setup  based  on  available 
anthropometric  data  that  he  had  applied  to  the  workstation.  This  project  was  spurred  by  his 
observations  that  the  crews  would  sometimes  not  be  able  to  reach  or  see  important 
components  of  the  workstation.  Simply  by  applying  an  existing  database  of  reach  data  with 
data  about  forces  required  to  operate  certain  switches  or  knobs  and  dials,  Earl  came  up  with  a  more 
ergonomically  sound  workstation. 

In  order  to  test  the  new  design  concepts  Earl  recruited  crews  to  mn  as  subjects  in  highly 
realistic  mission  scenarios  which  consisted  of  real  mission  briefings,  the  mission  itself,  and 
the  debrief,  in  order  to  provide  the  crews  with  the  most  realistic  experience  possible.  At  the  time, 
SAC  provided  the  details  of  the  up-to-date  threat  types  from  Vietnam  in  order  to  challenge  the 
crews  and  make  them  act  as  if  they  were  really  there.  The  crews  reported  that  after  about  thi^ 
minutes  they  had  forgotten  they  were  in  a  simulator,  and  also  reported  that  the  realistic  missions 
provided  them  with  better  training  than  some  of  the  simulator  training  conducted  in  the  wings. 

Earl's  discoveries  were  sometimes  frowned  upon  by  SAC.  Earl  would  discover  what  really  went 
on  in  the  aircraft  rather  than  what  doctrine  dictated  or  what  SAC  thought  was  happening.  He  was 
able  to  guarantee  the  subjects'  anonymity  and  confidentiality  while  they  were  being  tested,  so  the 
crews  acted  as  they  would  in  the  real  aircraft.  This  brought  credibility  and  respect  to  Earl's  findings 
from  the  crews  and  from  SAC. 


One  example  of  this  kind  of  discovery  was  evident  from  a  study  that  utilized  new  threat  types 
that  intelligence  had  only  recently  released  to  Earl  and  to  the  wings.  The  wings  had  been 
given  the  threat  type  information  to  disseminate  tp  the  crews  but,  due  to  the  amount  of 
administri via  they  were  forced  to  do,  the  information  was  locked  in  a  safe  and  a  general 
announcement  was  made  that  the  iiiforniation  was  available.  The  crews  never  read  the 
information.  When  Earl  presented  these  threats  some  time  later  in  an  evaluation,  the  crews 
were  not  prepared  to  act  appropriately.  Some  time  after  that  evaluation,  Earl  heard  on  the 
grapevine  that  SAC  had  instructed  the  wings  to  concentrate  on  operational  details  above  all 
administri  via.  Of  course  these  changes  weren't  attributed  to  Earl's  evaluation  findings,  but 
they  did  occur  very  shortly  after  the  evaluation.  (SAC  might  give  you  a  different  story.) 


A  similar  facility  was  subsequently  set  up  for  the  B-52  offensive  workstation,  and  B-52  flight 
deck,  with  similar  data-collection  capabilities. 

The  Next  Generations 

In  1983,  SAC  asked  Earl's  group  to  build  a  simulator  facility  for  the  new  B-1  bomber  which 
was  under  construction  but  for  which  there  was  no  training  device  as  yet.  He  built  two 
simulators  for  them  to  serve  as  interim  trainers  while  the  production  trainers  were  being  produced. 
While  these  two  simulators  were  being  fielded  Earl  built  another,  higher  fidelity,  defensive 
simulator  for  the  B-1  and  they  began  a  similar  operation  for  the  B-1  as  he  had  for  the  B-52. 
Subsequently,  the  two  fielded  units  were  returned  to  the  lab  and  served  as  an  experimental  platform 
for  all  B-1  crew  positions.  Earl's  group  is  currently  working  on  similar  devices  for  the  B-2,  which 
is  to  be  Earl's  legacy  to  the  Armstrong  Laboratory. 
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Earl's  work  has  always  been  highly  regarded  by  the  crews,  the  SPOs,  SAC,  and  the  Armstrong 
lab.  His  approach  to  his  job  is  based  on  being  able  to  understand  and,  in  some  cases,  do  the  job  of 
his  user  group.  This  has  required  hours  of  sitting  with  the  crews,  working  with  the  crews,  talking 
to,  and  debriefing  the  crews:  getting  into  their  heads  and  being  one  of  them.  For  this  he  has  been 
rewarded  with  the  trust  and  respect  of  all.  In  return  he  has  provided  the  Air  Force  with  state-of-the- 
art  research  and  testing  facilities  as  well  as  numerous  modifications  which  have  been  implemented 
in  the  aircraft  modification. 

Earl’s  keys  to  success?  Keys  to  overcoming  the  stumbling  blocks  and  barriers,  to  showing 
initiative?  He  claims  he  came  across  few  stumbling  blocks.  The  reason  may  have  been 
because  of  his  self-starter  attitude  of  avoiding  going  to  high  level  help,  taking  responsibility 
for  the  success  of  his  own  projects,  and  being  given  the  freedom  to  avoid  getting  caught  up  in  the 
red  tape  that  it  requires  to  get  things  approved  in  advance.  He  has  found  it  is  easier  to  ask 
forgiveness  than  permission,  but  he  has  earned  that  privilege  from  years  of  dedication  to  the 
betterment  of  the  bomber  crews'  workstation,  and  from  the  success  of  his  endeavors. 


Earl  estimates  it  took  him  about  two  years  to  be  productive.  Two  years  of  active  data  analysis, 
sitting  in  front  of  videos  of  the  crews  performing  their  day-to-day,  minute-to-minute  tasks. 
Two  years  of  talking  to  the  crews  in  front  of  the  video,  asking  them  what  they  were  doing, 
why  they  were  doing  it,  what  had  prompted  them  to  do  that.  This  active  data  analysis  was  key 
to  understanding  the  domain  and  later  being  able  to  make  informed  choices  for  ^st, 
evaluation,  and  modification,  This  close  interaction  with  the  crews  and  "getting  into  their 
heads "  is  seldom  done  by  short  term  contractors  and  is  also  difficult  for  the  Armstrong  lab 
people  to  do  nowadays; 


Summary 

What  if  Earl  hadn't  been  around?  Would  there  have  been  a  longstanding  simulation  and 
testing  facility  developed  for  the  bombers?  Would  there  now  be  a  high  fidelity,  sophisticated 
data  collection  and  human  factors  research  device  for  the  B-52,  B-1,  or  B-2?  Are  the  crews 
flying  in  safer  aircraft  with  more  effective  control  panels  and  displays?  Earl  wouldn’t  stretch 
it  that  far,  "You  can't  say,  because  the  'successes’  have  never  been  tried  any  other  way;  so 
you  can't  test  the  alternative.  But  many  of  the  changes  have  been  made  on  the  drawing  board 
rather  than  with  the  welding  torch." 
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The  Case  of  the  Fat  Cursors: 
The  B-2  Synthetic  Aperture  Radar 

An  Interview  with  Earl  Sharp 


In  1969  Earl  Sharp  chanced  on  an  old  friend  looking  at  some  videotapes  of  the  defensive 
workstation  of  the  B-52.  This  fortuitous  encounter  led  Earl  to  take  an  interest  in  the  Electronic 
Warfare  Officer's  (EWO)  workstation  which  has  led  to  a  20-year  commitment  to  improving  the 
design  of  the  crew  station  in  the  latest  Air  Force  high  tech  bombers,  including  the  B-1 ,  B-lb,  and 
currently  the  B-2. 

Earl  has  been  responsible  for  the  development  of  some  of  the  most  realistic  bomber  simulation  and 
testing  facilities  for  human  factors  research  in  the  world.  Having  persuaded  the  powers-that-be  in  a 
task  force  headed  by  Pentagon  Air  Staff  and  made  up  of  Strategic  Air  Command,  Air  Force 
Systems  Command,  several  Air  Logistics  Commands,  and  many  Air  Force  engineering 
organizations,  to  give  him  $270,000  and  a  B-52  training  device  in  1971,  Earl  set  about  putting 
together  a  high  fidelity  simulation  facility.  The  facility  had  the  capability  to  collect  performance  data 
from  every  thrown  switch,  twisted  knob,  and  CRT  display.  This  facility  has  since  produced 
valuable  data  which  has  provided  the  impetus  for  many  crew  station  and  display  design  changes 
for  the  B-52,  as  well  as  being  the  template  for  a  similar  B-1  facility,  and  Earl's  legacy,  a  B-2  test 
and  evaluation  facility. 

Earl's  philosophy  has  always  been  to  put  the  aircrew  member  in  as  realistic  a  scenario  as  possible 
and  see  what  he  really  does.  This  required  the  development  of  a  realistic  simulator  with  realistic 
scenarios.  Earl  has  become  so  respected  in  the  bomber  community  that  he  is  trusted  with  the  most 
up-to-date  threat  types  for  his  simulations,  and  is  provided  with  the  means  to  sit  in  the  wings  and 
talk  turkey  with  the  guys  to  whom  he  dedicates  his  work.  This  respect  is  carried  through  from  the 
crews,  to  the  SPO,  th^e  Human  Engineering  division,  and  Strategic  Air  Command. 


During  a  project,  daily,  and  sometimes  nightly, 
communications  are  set  up  with  the  friends  he 
has  made  in  these  organizations  in  order  to  talk 
about  the  problems  and  issues  at  the  heart  of  the 
problem,  and  come  up  with  solutions  that  will 
make  a  difference  to  the  guys  that  fly  the 
missions. 

Being  able  to  put  himself  in  the  crew's  shoes  has  enabled  Earl  to  develop  an  empathy  and 
understanding  of  the  aircrew  member's  task,  and  therefore  to  come  up  with  realistic  and  functional 
redesign  solutions  to  the  problems  given  to  him  by  the  SPO.  The  latest  problem  tackled  by  Earl  and 
his  group  was  presented  by  the  SPO  as  a  target  designation  accuracy  problem  due  to  the  thickness 
of  the  cursor  in  the  Synthetic  Aperture  Radar  of  the  B-2. 


Earl  has  become  so  f^lihf  with  the  domain 
that  the  avionics  group  at  S  AC  HQ  required  that 
all  modificatiohs  to  the  def^hsive  workstation 
go  through  Earl's  group  before  it  go  into  the 
aircraft.  The  avionics  group  believed  that  Earl 
knew  the  doniain  bettei'lhah  they  didv 
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The  Case  of  the  Fat  Cursors 


In  December  1993,  Sharp  and  his  colleagues  received  a  service  report  from  the  Crew  Test  Force 
(CTF).  The  service  report  outlined  a  problem  with  the  Synthetic  Aperture  Radar  (SAR)  cursor.  It 
was  too  fat  for  the  aircrew  member  to  position  accurately.  The  aircrew  member  needed  to  be  able  to 
position  the  cursor  within  one  pixel  of  the  target,  but  the  cursor  was  three  pixels  wide  so  it  would 
sometimes  obscure  the  actual  target.  The  CTF  and  B-2  SPO  thought  that  the  solution  was  probably 
to  make  the  cursor  only  one  pixel  wide,  but  wanted  Earl  to  evaluate  a  solution  to  demonstrate 
performance  improvements.  Earl's  team  flew  into  action.  They  began  to  get  the  facts  about  the 
cursor,  they  talked  to  the  crews,  the  CTF,  and  commiserated  with  the  SPO  over  the  phone.  Earl's 


This  particular  service  report  came  without  prior  notification,  but  Earl  keeps  his  ear  to  the  ground 
when  it  comes  to  potential  problems.  Regular  contact  with  the  B-2  SPO,  the  CTF,  and  the  crews 
by  phone  and  by  being  on  the  e-mail  circuit  enables  him  to  keep  up  with  the  latest  developments 
and  problems  as  they  arise. 

team 

uncovered  a  number  of  other  issues.  The  crews  had  also  reported  problems  with  the  mechanization 
of  the  cursor  and  the  positioning  thumb  controller  which  contained  delays  and  deadband  which 
hindered  cursor  positioning. 

Having  brainstormed  to  produce  as  many  alternative  designs  as  possible,  they  narfowed  the 
options  down  to  a  testable  number  and  began  to  prototype  them  on  the  system  to  see  what  they 
looked  like  and  how  they  worked.  The  goal  of  the  evaluation  was  to  prove  that  an  alternative 
design  would  provide  solid  performance  advantages  over  the  existing  cursor  design. 


One  of  Earl's  jobs  is  to  help  the  SPO  redefine  the  problem.  The  human  factors  knowledge 
combined  with  an  intimate  understanding  of  the  crew  arid  their  use  of  the  SAR  enabled  Earl  and 
his  team  to  think  about  the  cursor  not  merely  as  a  cross  on  a  screen  but  as  a  pointer  that  must  be 
directed  quiekly  and  aecurately  within  a  cluttered  radar  display.  This  provided  Earl's  team  with  a 
unique  view  of  the  cursor  that  was  not  so  clearly  understood  %  the  SPO  or  CTF,  enabling  them 
to  come  up  with  useable  solutions.  In  this  case  there  was  not  enough  time  to  deal  with  the 
positioning  mechanism,  but  Earl  pointed  this  out  as  another  issue  to  be  resolved  in  the  future. 
Earl  always  likes  to  give  more  than  he's  asked  for  on  a  project. 


Earl's  ability  to  know  what  the  crews  will  want  is  based  on  his  team's  vast  experience  base  and 
cooperation  with  the  bomber  crews.  His  team  has  specialized  in  the  defensive  aspects  of  the 
bomber's  task.  A  navigator  colleague,  Jim  LaSalvia,  provided  expert  knowledge  about  the  SAR 
system.  Earl  knows  the  aircrew  member’s  job,  the  day-to-day  and  minute-to-minute  tasks,  the 
equipment,  the  people. 


With  the  potential  of  missing  alternative  designs,  Earl's  philosophy  is  that  it's  easier  to  eliminate 
options  once  they've  been  generated  than  to  generate  new  ones  later  in  the  process. 
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Towards  the  end  of  January,  the  options  had  been  narrowed  down  to  a  useable  number,  and  the 
team  was  about  to  start  testing  in  a  few  days'  time  when  they  discovered  that  what  they  assumed  to 
have  been  a  black-and-white  cursor  on  a  green  SAR  screen  turned  out  to  be  a  black-and-white 
cursor  on  a  black-and-white  screen.  All  the  previous  versions  of  the  SAR  that  the  team  had  seen 
were  green  screens. 

The  whole  problem  had  changed,  now  that  they  realized  the  cursor  was  too  small  and  got  lost  in 
the  clutter  of  the  screen  imagery.  Not  only  did  they  have  to  make  it  thinner,  now  they  had  to  make 
it  more  easily  seen  and  less  easily  lost.  A  rapid  brainstorming  session  took  place  and  new 
candidates  were  suggested  and  rejected  or  accepted  based  on  accuracy  of  placement,  and  now  also 
conspicuity.  Two  candidate  cursors  were  kept  for  evaluation  against  the  original  cursor  by  the 
crews  using  simulated  displays  (see  Figure  1). 


Earl's  team  brainstormed  with  the  CTF,  the  B-2  crews,  and  the  SPO  in  order  to  come  up  with 
potential  redesigns.  This  process  is  vital  to  the  generation  of  as  many  options  as  possible,  and  to 
the  development  of  a  team  understanding  and  a  mutual  responsibility  for  and  investment  in  the 
end  product. 


I 


I 

I 


I 


I 
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(a)  Original  SAR  Cursor  (b)  Hammerhead  Cursor 


(c)  T-  Bar  Cursor 


Figure  1.  The  three  cursor  designs  used  in  the  evaluation. 


By  February  the  study  was  ready  to  be  mn.  The  candidate  cursors  were  incorporated  into  the 
simulator  in  a  realistic  target  designation  scenario.  Subjects  with  SAR  experience  were  recruited 
and  run  through  the  simulations  using  the  three  different  cursor  types. 
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The  crews  AVere  mom  than  h^py  to  help  Eari  out  in 
used  were  rnOre  challenging  and  up-to-date  than  soi 

evaluating  a  hew  design.  The  simulations  he 
Tie  of  the  training  in  the  Wings.  Earl's 

reputation  and  rapport  with  the  crews  has  enabled  h 

im  to  rely  on  diem  for  their  support  in 

running  ;his  studies; 

By  March  the  evaluation  was  complete  and  Earl  was  able  to  present  the  results.  The  T-bar  cursor 
had  provided  the  clearest  advantage  in  positioning  accuracy  and  conspicuity.  Within  two  to  three 
weeks  of  the  end  of  the  evaluation  Earl  was  able  to  provide  the  preliminary  results. 

The  temptation  at  this  point  may  have  been  to  present  the  data  to  the  crews  since  they  were  the 
primary  user.  However,  Earl  knows  the  system;  Don't  bite  the  hand  that  feeds  you!  His  first 
presentation  is  always  to  the  primary  sponsor  of  the  project,  in  this  case  the  SPO,  followed  by  the 
other  interested  parties  including  the  CTF,  operational  command,  ACC,  and  the  bomber  wings 
themselves.  Often  nowadays  these  presentations  are  to  multiple  interested  parties  simultaneously. 


The  timing  of  the  pi 

•esentatiOn  is  also  a  criticai  issi 

le.  Even  though  Earl  wants  to  get  the 

information  to  the  S 
deadline,  he  wants  t 

Pf)  and  crews  as  quickly  as  pe 
o  make  sure  his  data  are  solid 

>Ssible  in  Order  to  meet  the  modification 
and  reliable  before  he  presents  them; 

A  technical  report  was  also  written  by  Earl's  team,  describing  the  evaluation,  and  submitted  to  the 
Armstrong  Laboratory  (AL/CF-TR-1994-0020). 

The  modification  to  the  SAR,  based  on  the  recommendations  from  Earl's  study,  were  reviewed  by 
Northrop  for  engineering  considerations  and  equipment  compatibility.  Several  months  later  they 
were  incorporated  in  the  next  modification  of  the  B-2.  An  appreciation  for  the  acquisition  cycles 
and  stages  of  development  is  needed  in  order  to  be  able  to  bring  the  project  in  under  the  deadline 
that  precedes  the  preliminary  design  review. 


With  the  exception  of  the  tech  report  the  project  was  informal  in  that  there  was  no  administrative 
review  from  AL,  no  documentation  or  reports  other  than  Earl’s  own  paper-trail  of  memos  and 
messages.  These  kinds  of  informal  interactibhs  defy  schedulihg.  They  peritiit  rapid  inforhiation 
gathering  and  give-and-take  without  going  through  lengthy  administrative  red  tape, 


Earl's  primary  concern  is  for  the  bomber  crews  and  he'll  do  whatever  he  can  to  support  them  and 
help  them  in  any  way.  His  perseverance  in  the  face  of  adversity  is  based  on  his  discovery  of  an 
enthusiastic,  imaginative,  and  unique  group  of  users  who  were  eager  to  be  recognized  and  listened 
to.  Earl  saw  this  opportunity  and  it  has  led  to  a  20-year  relationship  that  has  fueled  his  efforts  and 
produced  numerous  successes  in  work  and  friends  for  life. 

One  reason  why  the  SPO  relies  on  Earl  and  his  team  is  that  they  know  he'll  get 
the  job  done,  and  done  on  time.  They've  worked  closely  together  before,  trust 
each  other,  and  know  each  others'  turn-around  times.  If  Earl  had  not  been  able 
to  act  quickly  to  analyze  and  present  the  results  within  the  time  cycle  of  the 
modiHcation,  money  would  have  been  wasted  later  for  an  on-line  modification 
or  the  new  cursor  may  not  even  have  been  implemented  until  a  later 
modification. 
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Summary 


The  SPO  had  suggested  a  simple  fix  to  the  fat  cursor  problem:  Make  it  skinnier.  They  wanted  Earl 
to  provide  the  performance  data  to  validate  the  change.  In  assessing  the  problem,  Earl's  team  had 
redefined  the  problem  and  come  up  with  workable  solutions  to  the  real  problems  faced  by  the 
crews  when  using  the  SAR  cursor.  Not  only  was  the  cursor  too  thick  to  accurately  designate  a 
target,  but  if  it  was  merely  made  thinner,  it  would  also  have  been  difficult  to  find  it  in  the  clutter  of 
the  display. 


The  domain  knowledge  and  understanding  of  user  requirements  held  by  Earl's  team  enabled 
them  to  imagine  the  itnplications  of  the  different  cursor  redesign  options.  They  were  thus  able  to 
provide  realistic  alternative  cursor  designs  for  evaluation  and  implementation, 


Earl's  team  saved  the  SPO  time  and  money  by  identifying  the  critical  issues,  evaluating  only 
workable  alternatives,  avoiding  the  implementation  of  an  unsatisfactory  cursor,  and  avoiding  the 
necessity  for  a  further  modification.  Had  the  small  skinny  cursor  been  implemented,  the  crews 
would  have  had  a  harder  time  successfully  completing  their  missions.  Eventually  the  complaints 
would  have  mounted  up  and  a  further  fix  would  have  to  have  been  made. 


Flying  Rhead  of  the  High-G  Cockpit 

An  Interview  with  Phil  Kulwicki 


Applied  research  is  sometimes  like  hitting  a  moving  target.  You  have  to  aim  where  you  think  the 
target  is  going  to  be,  rather  than  where  it  is.  Phil  Kulwicki  was  able  to  anticipate  human 
performance  problems  with  the  high-G  cockpit,  and  prepare  solutions  before  they  were  needed. 

The  Other  Side  of  Gravity 

Kulwicki  started  working  on  the  human  performance  implications  of  the  high-G  cockpit  in  1969. 
Prior  to  that  time  he  had  been  on  the  other  side  of  gravity.  After  graduating  from  the  University  of 
Detroit  in  aeronautical  engineering  in  1964,  he  began  working  in  the  Armstrong  Laboratory’s  Crew 
Stations  Branch  as  part  of  its  zero  gravity  research  program.  During  the  1960s,  the  Branch  was 
investigating  man-in-space  issues  as  the  era  of  spaceflight  dawned.  Important  parts  of  the  work 
involved  flight  testing  (using  C-47,  C-131B,  and  KC-135A  test  aircraft)  where  various  human 
performance  experiments  were  perf^ormed  in  the  airplane's  large  cargo  bay.  During  that  time 
period,  the  Laboratory  also  supported  NASA's  man-in-space  program  by  providing  zero  gravity 
familiarization  training  for  the  astronauts,  and  Phil  served  as  one  of  the  instructors.  This  was  the 
astronaut's  first  introduction  to  zero  gravity  "free-floating,"  experiencing  weightlessness  for  25-30 
seconds  during  the  airplane's  parabolic  maneuver  trajectory. 

After  completing  the  Master  of  Science  curriculum  in  aeronautical  and  astronautical  engineering  at 
The  Ohio  State  University,  Phil  rejoined  the  lab  in  1968  at  a  time  when  the  Air  Force  investment  in 
R&D  for  manned  space  applications  was  reduced  as  the  work  transferred  to  NASA.  Therefore,  he 
turned  his  attention  to  understanding  how  new  design  technology  would  affect  aircraft  pilots. 

The  Challenge  of  High-Performance  Airplanes 

In  1969,  Kulwicki  got  a  new  boss,  fighter  pilot  Major  Dick  Ravenelle.  Kulwicki  and  Ravenelle 
were  drawn  into  discussions  about  a  puzzle.  Between  them,  they  covered  the  pieces  of  aeronautical 
engineering,  human  performance,  recent  technology,  and  fighter  tactics.  The  puzzle  they  were 
trying  to  solve  was  how  pilot  performance  was  going  to  be  affected  by  the  new  high-performance 
capaWlities  of  the  next  generation  of  fighter  aircraft,  typified  by  the  highly  maneuverable  F-15  and 
F-16  fighters. 

These  tentative  explorations  set  the  stage  for  the  next  quarter  of  a  century  in  Kulwicki’ s  career  with 
the  Armstrong  Laboratory,  which  combined  laboratory  research  in  crew  station  design,  contract 
research  with  most  of  the  major  aircraft  and  avionics  companies,  and  applications  of  state-of-the-art 
technology  to  push  the  limits  of  the  pilot-aircraft  system. 

The  F-15  Eagle  could  sustain  9  Gs  in  a  level  turn  at  combat  weight.  Previous  generations  of  fighter 
aircraft  had  neither  the  airframe  design  nor  the  engine  power  to  operate  in  such  an  environment. 
How  well  would  pilots  withstand  such  forces  and  how  well  could  they  perform?  Before 
addressing  those  issues,  the  first  puzzle  was  whether  this  was  a  problem  worth  considering.  Just 
because  the  F-15  could  sustain  those  forces  did  not  mean  that  pilots  would  be  pulling  9  Gs  in 
combat.  There  is  no  sense  in  solving  a  problem  that  might  not  be  real. 
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The  Problem  is  Real 


Kulwicki  and  Ravenelle  worked  on  the  high-G  cockpit  research  program  from  1969  to  1972 
as  an  unfunded  in-house  project.  They  didn't  receive  much  in  the  way  of  laboratory  funds,  but 
they  could  devote  their  in-house  labor  to  the  project.  The  task  they  set  themselves  was  to  estimate 
whether  pulling  high-G  turns  in  aerial  combat  provided  much  benefit.  They  figured  that  if  the 
benefit  was  great  enough,  pilots  would  be  crazy  enough  to  maneuver  in  that  G-force  envelope. 

And  that  would  mean  that  they  better  start  thinWng  about  how  to  provide  sufficient  G-protection. 

In  order  to  continue  Armstrong  Laboratory  funding,  they  had  to  find  a  "customer"  for  the  project, 
an  organization  outside  of  the  Human  Engineering  Division.  Kulwicki  and  Ravenelle  contacted  the 
appropriate  people  at  the  Aeronautical  Systems  Division  at  WPAFB,  in  the  development  planning 
community.  There  was  agreement,  given  the  new  cockpit  concept's  intuitive  appeal,  that  the 
previous  design  philosophy  (of  conventional  fighter  maneuvering  load  limits)  was  too  restrictive 
and  that  the  high-G  cockpit  could  be  a  key  to  the  new  design  freedom. 

In  order  to  help  justify  funding  by  Armstrong  Laboratory,  ASD  issued  a  formal  Technical  Need 
document.  Getting  the  technical  need  was  no  easy  feat,  but  after  eight  months  of  discussions  and 
selling  the  idea,  the  Technical  Need  was  issued  in  1970.  The  Technical  Need  was  highly  regarded 
by  ASD,  and  the  laboratory's  work  was  given  high  importance  ratings  and  high  progress  ratings  in 
the  annual  Technical  Need  reviews.  The  Technical  Need  also  proved  to  be  instrumental  in  gaining 
and  sustaining  industry  interest  in  the  new  concept.  For  Kulwicki  and  Ravenelle,  getting  the 
Technical  Need  issued  meant  they  had  the  freedom  they  needed  to  study  the  puzzle  of  whether 
high-G  maneuvering  would  result  in  tactical  advantages. 


The  discussions,  collaboration,  and  cooperation  with  ASD  were  instrumental  in  achieving  the 
tech  need.  ASD's  input  was  sought  as  to  ho'w  the  project  should  proceed,  identifying  the 
boundaries  of  the  work,  and  convincing  ASD  that  the  pay-offs  and  benefits  would  be  what  they 
claimed  they  would  be.  The  process  of  obtaining  the  teCh  need  was  considered  to  be  much 
smoother  because  of  the  identification  of  a  user-group  at  Wright-Patt. 


At  that  time,  digital  computer  models  of  relative  aircraft  maneuvering  were  being  constructed,  and 
Kulwicki  and  Ravenelle  decided  to  use  such  a  model  in  building  the  case  for  a  high-G  cockpit.  The 
Rand  Corporation  had  developed  a  computer  program  that  was  named  TACTICS  n  after  its 
intended  use  as  a  simulation  tool  to  help  analyze  the  tactics  of  air  combat.  After  contacting  the  Rand 
developers,  they  obtained  the  TACTICS  11  model  for  free,  and  installed  it  on  a  laboratory 
computer.  The  team  got  some  more  free  help,  because  they  were  joined  during  this  period  by 
Captain  John  Lyons,  a  graduate  electrical  engineer  from  AF'IT.  They  began  to  analyze  the  potential 
operational  payoffs  of  flying  high  acceleration  maneuvers. 

The  laboratory  team  used  the  Rand  computer  model  to  estimate  the  increased  combat  effectiveness 
of  next  generation  fighter  aircraft,  the  F-15  and  F-16.  That  was  done  by  simulating  air-to-air 
maneuvering  dogfights  with  a  range  of  aircraft,  varying  the  airplane's  structural  limit  load  factor. 
The  aircraft  were  "flown"  against  one  another  in  a  large  number  of  separate  air  combat 
engagements,  using  missile  and  gun  firing  opportunities  as  measures  of  success.  Typically,  one  of 
the  aircraft  was  programmed  with  maneuvering  capabilities  above  the  previously  accepted  pilot  G- 
tolerance  limit  of  7.5  Gs,  and  the  resulting  (maneuver  and  combat)  superiority  over  its  adversary 
was  demonstrated  by  achieving  weapon  firing  opportunities  earlier  in  the  simulated  engagement. 
Even  after  paying  the  price  of  extra  weight  for  the  aircraft  stmcture  needed  to  increase  the 
maneuver  load  factor,  Kulwicki,  Ravenelle,  and  Lyons  demonstrated  clear  evidence  of  the 
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projected  operational  payoff.  The  first  part  of 
the  puzzle  was  solved.  By  moving  above  the 
G-tolerance  limit  of  7.5  Gs,  pilots  would  gain 
a  clear  advantage. 

The  team  presented  their  results  at  a  classified 
symposium  in  1972.  Immediately,  the  work 
caught  the  attention  of  the  fighter  development 
community  in  both  the  Air  Force  and  industry. 


The  work  was  later  extended  in  an  AFIT 

master's  thesis  by  Major  Dean  Vikan,  which  involved  the  parametric  evaluation  of  a  family  of 
aircraft  designs  built  around  the  high-G  cockpit  concept,  using  the  same  TACTICS  11  software. 
These  computer  projections  of  capability  provided  the  initial  ammunition  with  which  to  sustain  the 
ASD's  Technical  Need,  and  to  help  to  justify  the  subsequent  laboratory  funding  for  concept 
development. 

Kulwicki  had  also  arranged  for  some  converging  evidence.  After  the  Technical  Need  was  issued  in 
1970,  Kulwicki  arranged  for  the  first  R&D  contract  in  the  project,  a  contract  with  the  McDonnell 
Aircraft  Company  (MCAIR).  At  the  time,  MCAIR  had  just  developed  a  high-  fidelity  flight 
simulation  facility  with  the  then  state-of-the  art  scene  generation  capability.  Such  an  engineering 
simulator  can  be  used  to  rigorously  evaluate  the  aircraft  and  cockpit  during  development,  and  was 
well  suited  to  study  air  combat.  MCAIR's  advanced  engineering  project  staff  was  also  working  on 
F-15  design  upgrades  for  ASD.  MCAIR’s  initial  contract  work  involved  both  cockpit  design  layout 
(i.e.,  design  and  installation  within  vehicle  engineering  constraints)  and  new  estimates  for 
projected  combat  payoff  using  its  computer  simulation  model  resembling  TACTICS  11. 

Can  Pilots  Flv  the  High-G  Maneuvers? 

Once  it  became  clear  that  high-G  maneuvers  were  effective,  the  second  puzzle  was  how  to  help 
pilots  fly  these  maneuvers  without  blacking  out.  The  F-15  cockpit  uses  a  conventional,  upright 
ejection  seat  and  a  bubble  canopy,  giving  the  pilot  the  access  to  good  external  vision  that  is  needed 
for  air  combat.  However,  even  with  G-suit  protection,  the  upright  seated  position  also  makes  the 
human  most  susceptible  to  blackout  from  the  high  G-forces  that  are  available  at  the  pilot's 
command.  The  G-suit  protected  pilot  was  known  to  be  able  to  withstand  about  7.5  Gs.  However, 
the  advanced  F-15,  once  it  had  spent  part  of  its  fuel,  could  pull  9  Gs  at  the  drop  of  a  hat. 

The  F-16  was  another  case.  Despite  the  initial  claims  of  improved  G-tolerance  in  the  F-16,  with  its 
30  degree  seat  reclining  angle,  subsequent  centrifuge  data  proved  that  the  seat  recline  didn't  add 
much.  The  seat  was  only  more  comfortable,  to  the  point  of  black-out!  From  these  studies  emerged 
an  extension  of  the  G-tolerance  database  which  serves  now  as  a  reference  for  cockpit  and  aircraft 
design. 

The  Articulating  Seat 

Kulwicki  began  to  wonder  what  would  happen  if  the  seat  could  be  reclined  back  even  further.  An 
“articulating  seat”  was  designed  which  allowed  the  pilot  to  change  the  angle  of  recline  during  the 
operational  mission,  such  as  upon  entry  to  a  combat  zone.  The  seat  could  be  set  more  upright  at 
t^e-off  and  during  routine  aspects  of  flight,  for  example,  and  reclined  back  to  an  optimal  angle  in 
cases  where  maximum  G-tolerance  was  required.  This  would  expand  the  pilot's  G-tolerance  (and 
ability  to  think  and  react).  The  articulating  seat  actually  raised  the  pilot's  torso  and  legs  rather  than 


The  first  step  to  building  the  credibility  of  the 
research  program  was  die  use  of  the  Rand  air 
combat  sihiulatibn,  TAGTIGSp.  Despite 
factions  in  the  engineeiing  community  that 
were  skeptical  of  the  potential  benefits  of  using 
Cpmputef  models  to  simulate  dynarhib 
environttients,  KulWibki,  Ravenelle  and  Lyons 
found  a  testbed  to  show  the  feasibility  of  their 
ideas  about  high  performance  aircraft, . . .  and 
the  potential  for  a  high  acceleration  cockpit. 
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simply  lowering  the  backrest.  This  was  done  to  preserve  the  important  out-of-cockpit  vision  which 
is  paranaount  for  air  combat.  Also,  by  using  an  electromechanical  insert  to  the  conventional  ejection 
seat  they  could  avoid  the  cost  of  developing  a  new  ejection  seat. 

MC air’s  work  was  so  promising  that  a  sole  source  follow-on  contract  was  established  with 
MCAIR  in  1973  for  man-in-the-loop,  interactive  combat  simulation  and  to  build  and  deliver  a 
variable  geometiy,  articulating  seat  to  investigate  the  human  G-tolerance  under  actual  G-forces  in 
the  centrifuge.  A  third  contract  was  awarded  to  MCAIR  in  1974  for  a  follow-on  air  combat 
simulator  evaluation.  Highly  experienced  fighter  pilots,  all  fighter  weapons  school  graduates  or 
instructors  from  the  Air  Force  and  Navy,  "flew"  hundreds  of  one-on-one  air  combat  engagements 
in  the  MCAIR  simulator,  providing  another  layer  of  data  testifying  to  the  concept's  promise.  The 
centrifuge  seat  was  first  used  at  Brooks  AFB  for  proof  testing  and  later  installed  at  Wright- 
Patterson  AFB  for  additional  research  in  human  G-tolerance  as  a  function  of  seat  position. 

Building  on  that  work,  additional  contracts  were  awarded  to  MCAIR  (for  the  F-15)  and  to  General 
Dynamics  (for  the  F-16)  from  1974  to  1976,  to  work  out  the  installation  details  for  a  prospective 
flight  demonstration.  Design  issues  were  worked  out,  such  as  the  placement  of  hand  controls, 
impact  on  cockpit  display  panels,  ejection  seat  concepts,  and  related  engineering  design  factors. 
Other  airframe  contractors  began  to  study  the  concept  with  their  own  funds.  From  1971  through 
1974,  Kulwicki  and  Ravenelle  visited  every  major  military  fighter  maker  in  the  country  to  discuss 
the  project  and  share  results.  Thus,  industry  was  also  developing  the  engineering  design  data 
needed  for  future  applications.  Preparations  were  nearly  ready  for  a  flight  demonstration. 

In  1974,  following  the  successful  work  conducted  in  support  of  the  ASD  tech  need,  the 
Flight  Dynamics  Laboratory  at  Wright-Patterson  AFB  became  convinced  that  the  work  could 
turn  into  an  advanced  development  (6.3)  project.  At  the  time,  Armstrong  Laboratory  did  not  have 
advanced  development  projects,  but  the  Flight  Dynamics  Laboratory  did  and  they  became 
interested  in  high-G  cockpit  flight  demonstrations  on  the  F-15  and  F-16.  Demonstrations  using 
both  aircraft  were  planned.  This  led  to  a  profitable  collaboration  with  Wright  Laboratories,  with 
Kulwicki  at  the  Armstrong  Laboratory  providing  technical  guidance  and  Ravenelle  moving  to 
Flight  Dynamics  Laboratory  to  coordinate  the  project's  transfer. 

An  advanced  development  project  was  approved  for  Flight  Dynamics  Laboratory  in  1976,  and 
considerable  engineering  was  performed.  But  this  part  of  the  story  ends  on  a  disappointing  note. 
The  actual  flight  demonstrations  were  not  performed  when  funds  for  the  flight  test  part  of  the 
advanced  development  were  removed. 

Better  Training  and  Equipment 

A  further  outcome  of  the  Laboratory's  interest  in  pilot  performance  in  high-G  environments  was  an 
increased  awareness  of  the  importance  of  the  design  of  personal  equipment  such  as  G-suits, 
oxygen  masks,  and  of  pilot  G-tolerance  training.  This  heightened  awareness  may  have  spurred  the 
suecessful  research  program  at  Brooks  AFB,  which  later  developed  personal  protective  equipment 
to  support  the  pilot  in  high-G  environments.  At  the  time  of  the  high-G  cockpit  project,  researchers 
at  Brooks  AFB  were  developing  the  M-1  straining  regimen  that  pilots  use  to  boost  their  G- 
tolerance,  and  were  using  the  human  centrifuge  to  train  fighter  pilots  to  detect  the  signs  of 
impending  blackout.  The  high-G  cockpit  work  preceded  those  separately  developed  advances  in 
life  support  equipment,  such  as  the  use  of  chest  counterpressure  and  positive  pressure  breathing. 
These  investigations  all  led  to  the  engineering  development  of  the  Combat  Edge  equipment,  which 
has  been  fielded  in  our  fighter  aircraft.  This  gear  helps  to  maintain  blood  in  the  head  and  upper 
body  to  forestall  blackout. 
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Summary 

It  is  likely  that,  without  the  work  conducted  at  the  Armstrong  Laboratory  by  Kulwicki  and 
company,  the  state-of-the-art  for  crew  station  design,  pilot  equipment,  and  pilot  training  for  high-G 
environments  would  not  have  matured  as  rapidly  as  needed,  to  keep  pace  with  the  emerging  breed 
of  highly  agile  fighter  aircraft.  Their  persistence  in  obtaining  the  objective  engineering  data  proved 
vital  to  the  credibility  and  feasibility  of  the  project,  especially  in  the  early  days.  The  safety  of  pilots 
and  overall  system  effectiveness  have  been  greatly  enhanced  because  the  issues  had  been 
anticipated  and  studied. 
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Targets  of  Opportunity:  The  Case  of  the 

Corrupted  Cues 

An  Interview  with  Phil  Kulwicki 


"Fighter  aircraft  exist  to  destroy  other  aircraft.  The  airplane  itself  may  be  considered 
only  a  weapons  platform  designed  to  bring  the  weapons  system  into  position  for 
firing....  All  requirements  of  a  particular  weapon  must  be  satisfied  simultaneously 
in  order  for  the  weapon  to  be  used  successfully." 

(Shaw,  1985,  p.  1) 

"The  air-to-air  gunnery  problem  is  a  difficult  one;  it  involves  hitting  a  moving  target 
from  a  moving  platform  with  projectiles  that  follow  curved  paths  at  varying 
speeds."  (Shaw,  1985,  p.7) 

During  times  of  revolutionary  change,  it  is  easy  to  overlook  parts  of  a  system  that  have  worked 
well  in  the  past,  but  will  be  inadequate  for  the  future.  This  was  the  problem  when  the  high 
acceleration  fighters,  the  F-15  and  F-16,  came  onto  the  scene.  The  cues  about  when  to  shoot 
changed  dramatically.  Fortunately,  Phil  Kulwicki,  and  his  colleagues  at  the  Human  Engineering 
Division  had  been  anticipating  these  difficulties  and  were  ready  with  solutions. 

High  Angle-off  Shots 

In  the  past,  fighter  pilots  had  to  judge  their  firing  opportunities  using  their  skill  and  experience, 
taking  into  account  multiple  characteristics  of  their  own  weapons,  their  own  flight  path  and  the 
maneuvering  of  the  enemy.  With  the  progression  of  technology,  gun  and  missile  firing 
opportunities  are  now  calculated  and  cued  using  algorithms  that  take  this  information  into  account. 
When  a  valid  shooting  opportunity  is  calculated,  a  "shoot"  cue  is  displayed  to  the  pilot  on  the  HUD 
indicating  a  high  probability  of  a  hit  if  the  weapon  is  released  at  that  point  in  time. 

To  aid  in  the  art  of  gunnery,  the  Lead  Computing  Optical  Sight  (LCOS)  was  developed  to  reduce 
the  mental  calculations  and  predictions  required  by  the  pilot  to  hit  his  adversary.  However,  with  the 
advent  of  low  wing  loading,  highly  agile  aircraft  such  as  the  F-15  and  F-16  in  the  mid-sixties  and 
early  seventies,  the  assumptions  on  which  the  LCOS  prediction  algorithms  were  based  became 
obsolete.  Rather  than  shooting  at  the  target  from  its  "six"  at  1  G,  the  pilot  was  now  having  to  take 
fleeting  shots  at  high  deflections  from  the  enemy  aircraft's  nose  or  tail.  The  algorithms  were  not 
designed  to  compute  the  required  information  for  these  high  angle-off  shots.  The  LCOS  was  both 
providing  invalid  shoot  cues  and  missing  some  valid  shooting  opportunities.  Even  small 
discrepancies  from  the  algorithm's  assumptions  could  result  in  improper  firing  cues.  This  problem 
was  not  confined  to  air-to-air  gunnery,  but  also  applied  to  firing  air-to-air  missiles,  a  problem 
which  was  later  addressed  by  Kulwicki's  group. 

Phil  Kulwicki  and  his  colleagues,  particularly  Dan  Eliason,  at  the  Armstrong  lab  became  interested 
in  this  problem  since  it  also  became  a  concern  for  both  the  Air  Force  and  Naval  air  combat 
communities.  A  system  that  provided  shooting  cues  that  were  based  on  invalid  assumptions  and 
were  poorly  displayed  to  the  pilot  was  operationally  and  economically  detrimental  to  both 
communities. 
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The  issues  involved  were  not  only  problems  of  prediction  but  also  problems  of  displaying  the 
information  to  the  pilot.  This  information  display  issue  particularly  sparked  Kulwicki's  interest  and 
led  to  a  three-year  effort  between  1977  and  1980  to  try  to  improve  the  fire  control  systems  in  the 
next  generation  of  agile  aircraft.  The  result  of  the  effort  was  improved  HUD  symbology  to 
accompany  the  recalculated  algorithms,  thus  improving  the  combat  effectiveness  of  the  F-15  and 


F-16. 


Finding  the  Problem 

In  1977  Kulwicki  attended  a  symposium  in  San  Diego  at  which  the  Air  Force  and  Naval  aviation 
communities  discussed  issues  concerning  the  evolution  of  air  combat.  This  conference  planted  a 
concern  for  display  issues  relating  to  the  cueing  of  valid  firing  opportunities  based  on  new 
assumptions  for  highly  agile  aircraft.  No  one  at  the  conference  appeared  to  be  tackling  the  problem, 
so  Phil  began  to  bounce  around  ideas  about  the  problem  in  terms  of  the  pilot-aircraft  interface  with 
Eliason  and  others  in  the  lab.  Eliason,  Phil's  branch  chief,  was  influential  in  the  early  stages  with 
his  background  as  an  F-105  pilot  and  his  interest  in  pursuing  the  idea  as  a  potentially  high  pay  off 
project  for  the  lab.  Kulwicki  had  also  been  in  contact  with  the  Avionics  lab  at  Wright-Patterson, 
which  had  already  been  involved  in  developing  fire  control  cueing  algorithms,  and  therefore  had 
some  technical  knowledge  of  the  problems  which  would  have  to  be  faced. 

Phil  recognized  an  opportunity  to  combine  the  Avionics  lab's  knowledge  of  the  algorithm  problems 
with  his  own  interest  in  the  display  issues  relating  to  human  performance  and  began  to  put  together 
his  ideas  for  a  proposal  for  the  AL  Annual  Program  Review  Board. 


Kulwicki  was  already  involved  with  the  fighter  aircraft  conimunity  after  the  AL's  involvement 
with  deveioping  crew  station  design  concepts  required  for  improved  pilot  performance  in  high-G 
environments.  This  involvement  had  forged  contacts  which  allowed  Riil  to  keep  his  ear  to  the 
ground  about  potential  cmw  station  and  display  design  issues  that  were  being  f^ed  by  the  pilots. 
A  further  important  result  of  this  earlier  work  was  contacts  with  the  aircraft  rnanufaeturers,  The 
ongoing  involvement  of  diese  industrial  contacts  was  inv&able  for  helping  Phil  establish  the 
credibility  of  the  lab's  design  concepts  and  getting  them  implemented  into  the  aircraft,  rather  than 
having  them  gather  dust  as  a  tech  report. 


The  Lab  Mission 


At  this  time  Phil  had  just  completed  another  project  and  had  sorne  time  to  pursue  other  projects  that 
caught  his  interest,  so,  following  the  Air  Force/Navy  symposium,  he  submitted  his  ideas  about  the 
gunnery  and  missile  cueing  problems  to  the  Annual  Program  Review.  The  work  that  he  proposed 
fell  within  the  charter  of  the  lab  and  had  already  been  identified  as  operationally  relevant  by  the  air 
combat  communities.  The  problem  had  also  been  bounded  by  the  operational  communities  and  the 
other  players  in  previous  related  efforts,  such  as  the  Avionics  lab  who  were  interested  specifically 
in  the  calculation  of  the  algorithms  which  would  drive  the  display  elements. 


Previopslyr  ih  1975,  Phil  had  written  a  White  Paper  which  outlined  a  new  lab  initiative  towards 
improving  crew  station  design  in  the  new  wave  of  fighter  aircraft.  The  Air  Force  had  envisioned 
the  obsolescence  of  the  F-1 1 1  and  F-4  to  be  replaced  by  aircraft,  such  as  the  F-15  and  F-16, 
with  capabilities  that  Would  stretch  the  pilot  to  the  limit,  both  physically  and  mentally.  Phil 
proposed  that  the  Armstrong  lab  should  become  involved  with  looking  into  the  man-machine 
interface  issues  that  he  recognized  would  affect,  and  were  already  affecting,  the  operational 
effectiveness  of  these  new  fighter  systems.  Thus,  when  the  fire  control  problem  proposal  was 
raised  at  the  annual  reView,  the  lab  had  already  been  primed  to  accept  this  type  of  project  and 
were  prepared  to  back  it  with  funding. 
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Assembling  the  Team 


The  proposal  of  such  a  project  involving  multiple  players  was  somewhat  unusual  within  the 
Armstrong  labs  at  that  time.  To  be  so  proactive  in  finding  issues  that  were  not  only  interesting 
from  a  research  and  development  perspective,  but  were  also  operationally  relevant  was  not  a 
common  occurrence  within  the  lab.  Contacts  forged  within  the  air  combat  community  through 
previous  work  allowed  Phil  to  identify  potential  opportunities  for  such  research. 


Phil's  proposal  included  the  need  to  interact  with  industrial  contacts  in  order  to  sensitize  them  to  the 
fire  control  cueing  issues  and  to  aid  the  transition  from  lab  research  to  the  actual  aircraft:  lessons 
learned  from  previous  projects  including  the  high-G  cockpit. 

Display  Concept  Development  and  Evaluation 

In  the  proposal  Phil  had  outlined  an  approach  utilizing  both  low-  and  high-fidelity  dynamic 
simulations.  Pilots  would  interact  with  the  simulations  and  give  their  judgments  of  when  a  good 
firing  opportunity  occurred  and  weapon  effectiveness  was  scored.  This  was  in  sharp  contrast  to 
similar  efforts  in  the  past  which  had  employed  static  pictures  of  the  different  fire  control  displays  in 
order  for  the  pilots  to  base  their  judgments  of  valid  firing  opportunities.  With  funding  provided  by 
the  Avionics  lab,  Phil  arranged  for  the  Hughes  Aircraft  Company  to  develop  and  evaluate  the  new 
fire  control  displays  using  the  dynamic  simulations. 


The  use  of  dynamic  displays  as  a  means  of  studying  how  firing  opportunities  were  assessed  was 
also  a  major  step  up  from  the  use  of  static  picture  representations  of  the  combat  situation.  The 
use  of  this  kind  of  technology  at  that  time  had  not  been  fully  embraced  and  the  decision  to  use 
these  techniques  was  not  without  question  within  the  lab. 


Based  on  the  pilots'  descriptions  of  what  information  they  would  use  to  calculate  a  valid  firing 
opportunity,  the  lab  was  able  to  produce  display  concepts  which  enhanced  the  pilots'  ability  to 
interpret  and  act  on  the  fire  control  cues.  The  fact  that  the  displays  were  dynamic  enabled  the  lab  to 
discount  artifactual  cues  from  the  static  displays  which  had  been  utilized  in  previous  efforts. 

During  1978  these  display  concepts  were  programmed  into  the  McDonnell-Douglas  and  General 
Dynamics  simulation  facilities  for  test  and  evaluation.  In  1979  they  were  evaluated  in  a  high  fidelity 
simulator.  The  new  fire  control  system,  including  both  the  improved  algorithms  and  the  new 
display  symbology,  was  found  to  improve  the  weapon  system's  effectiveness. 

SPO  Involvement 


The  display  recommendations  were  handed  off  to  the  SPO  and  subsequently  to  the  SPO's 
contractors,  the  companies  involved  in  the  development  and  evaluation  phases  of  the  project. 
Eventually  the  improved  fire  control  systems  were  implemented  in  the  F-15  and  F-16,  including 
the  new  computer  algorithms  and  improved  displays  for  gunnery  and  missile  control. 
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ftie  SPO  had  been  involved  from  early  on  in  the  project.  This  involvement  avoided  later 
complications  and  misunderstandings  about  the  key  decisions  that  were  made  in  the  design 
process.  Kulwicki  had  also  ensured  that  the  SPO's  contractors,  McDonnell-Douglas  and  General 
Dynamics,  were  brought  in  on  the  project  to  work  with  the  AL  so  that  when  the  final  stage  of 
implementing  their  ideas  came  about  the  key  players  were  already  in  place  and  up  to  speed. 


Summary 

This  relatively  short  project  for  Kulwicki  and  colleagues  resulted  in  identifying  a  problem  faced  by 
an  operational  community,  seeing  the  display  implications  that  were  part  of  the  problem,  and 
producing  a  new  display  configuration  that  improved  operational  effectiveness  in  the  new  wave  of 
USAF  fighter  aircraft. 

In  addition  to  the  obvious  impact  on  enemy  kills  and  subsequent  survival  rates  of  American  air 
crews,  the  work  had  an  economic  impact  by  curtailing  the  likelihood  of  wasted  shots  of  expensive 
missiles  fired  "out  of  launch  zone." 

Many  basic  and  applied  research  projects  are  designed  to  carefully  work  on  a  problem  for  several 
years  before  transitioning  it  to  the  next  step  in  the  development  cycle.  Here,  Kulwicki  was  able  to 
respond  rapidly  to  an  Air  Force  need,  and  to  use  his  networks  in  industry  and  in  the  SPOs  to 
quickly  inject  the  improved  symbology. 

In  a  way,  Kulwicki  was  taking  a  high  angle-off  shot  at  a  research  target.  He  quickly  noticed  the 
opportunity  and  rather  than  engage  in  a  lengthy  tail  chase,  he  was  able  to  lock  on  to  the  source  of 
the  difficulty  and  launch  a  solution  within  a  time  frame  that  ensured  impact. 

Reference 


Shaw,  R.  L.  (1985).  Fighter  combat:  Tactics  and  maneuvering.  Annapolis,  MD:  Naval  Institute 
Press. 


25 


On  Top  of  the  lUorld:  The  Case  of  the 
Confusing  Compasses 

An  Interview  with  Julien  Christensen’ 

One  of  the  challenges  facing  human  factors  professionals  is  to  take  what  they  can  see  and  convert  it 
into  data  that  will  convince  others.  This  incident  describes  how  Julien  Christensen  responded  to 
that  challenge  by  trying  to  improve  the  navigator's  tricky  task  of  navigating  in  the  polar  regions. 

This  incident  took  place  early  in  Christensen’s  career,  around  1947.  After  World  War  n,  Chris 
went  to  work  with  Paul  Fitts,  the  father  of  human  engineering  in  this  country.  Fitts  was  the 
director  of  the  Engineering  Psychology  Branch,  in  the  Aero  Medical  Research  Laboratory.  During 
Chris'  Air  Force  career  he  had  been  an  aerial  navigator  and  a  radar  bombardier.  This  background 
was  one  that  Fitts  felt  he  could  use. 

Arctic  Navigation 

"The  Air  Force  was  doing  reconnaissance  work  in  the  Arctic  (photographic,  and  weather 
reconnaissance).  They  were  having  lots  of  difficulties  with  their  navigation  because  most  of  the 
time  the  airplanes  were  too  far  up  to  home  on  the  north  magnetic  pole^  on  Prince  Edward  Island, 
north  of  Canada.  When  you  were  in  that  region,  the  precession^  rates  on  the  magnetic 
gyrocompass  were  just  unpredictable  and  unbelievable.  And  it  might  just  sit  there  and  kind  of 
slowly  whirl  around  the  entire  face  of  the  compass.  So,  a  big  team  was  necessary  just  to  do  the 
navigation.  Sometimes  the  missions  were  16-18  hours  long  and  they  would  often  have  two 
navigators  and  two  rear  operators.  You  had  a  team  of  about  four  people  trying  to  navigate.  When 
you  couldn't  do  celestial,  you  couldn't  see  the  ground,  and  you  were  out  of  radio  contact,  you  had 
a  very  difficult  problem." 

Having  been  a  navigator,  Chris  decided  to  fly  on  some  of  these  polar  missions  to  see  the  problems 
first  hand.  He  first  went  on  missions  in  the  mid-latitude.  He  went  to  Europe  a  few  times,  and 
Hawaii,  and  tried  writing  down,  narratively,  what  the  navigator  was  doing.  This  proved 
impossible.  "After  a  couple  hours  of  that  you've  had  it."  When  he  got  back  in  the  office  he 
thought,  why  not  just  sample?  That's  when  he  developed  what's  called  the  sampling  technique  for 
activity  analysis.  Later  he  found  that  the  British  were  doing  the  same  thing,  calling  it  Ratio  Delay." 


'This  interview  was  conducted  by  Marvin  Thordsen,  Klein  Associates  Inc.,  and  John  Flach,  Wright 
State  University.  This  account  is  based  on  an  account  written  by  John  Flach. 


^Because  the  compass  is  oriented  to  the  magnetic  pole,  allowance  must  be  made  for  the  difference  in 
location  of  the  geographic  and  magnetic  poles. 


^Precession  is  a  turning  or  tilting  of  the  axis  of  the  gyrocompass  due  to  applied  forces.  These  forces 
arise  from  aircraft  motions  and  cause  drifting  and  erroneous  indication. 


"This  technique  is  sometimes  referred  to  as  work  or  occurrence  sampling  and  is  discussed  in  most 
texts  on  work  design.  For  exanmle  see,  Konz,  S.  (1990).  Work  design:  Industrial  engineering. 
Worthington,  OH:  Publishing  Horizons,  Inc.  Ch.  (  p.  83  -  lC)(b.  or  Richardson,  W.J.  &  Pape,  E.S. 
(1982).  Work  sampling.  In  G.  Salvendy  (Ed.).  Handbook  of  Industrial  Engineering.  New  York: 
Wiley-Interscience. 
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Instead  of  trying  to  write  down  everything  that  the  navigators  were  doing  for  16-18  hours,  Chris 
decided  to  classify  their  activities  at  each  sampling  point,  and  he  sampled  every  five  seconds. 

He  developed  his  code  by  flying  missions  and  just  recording  what  they  were  doing.  If  the 
navigator  was  working  on  charts,  Chris  might  jot  down  "C."  Every  five  seconds  he  would  jot 
down  something.  This  worked  fairly  well.  Fortunately,  he  knew  exactly  what  the  navigator  was 
doing  any  time  he  looked  at  him. 

"Unless  you  understood  the  job,  you'd  have  real  trouble.  Aerial  navigation  is  a  very  specialized 
job.  It's  a  pretty  professional  job  involving  a  lot  of  calculations  and  log  keeping,  and  various 
instruments,  and  so  on.  Unless  you  were  pretty  familiar  with  the  job,  you'd  have  real  difficulty 
figuring  out,  at  times,  what  he  was  doing.  Is  he  drawing  a  line  or  plotting  position?  Is  he  about  to 
take  celestial  shots  and  doing  a  little  pre-computing,  or  what?" 

Chris'  categories  included  map  reading,  celestial  shooting,  radio  work,  E6B  computer  work, 
among  others.  There  might  have  been  a  dozen  categories,  including  rest  and  eating.  Afterwards,  he 
realized  that  one  could  use  a  sampling  technique  and  observe  an  entire  team.  "Suppose  there  was  a 
team  of  three.  The  first  five  seconds  Chris  would  observe  what  Operator  1  does.  The  second  five 
seconds  he'd  observe  Operator  2.  The  third.  Operator  3  and  fourth  back  to  Operator  1." 

He  tried  the  technique  out  experimentally  back  at  the  lab.  He  and  a  colleague,  Jim  Smithson,  went 
up  on  an  old  C-47.  Jim  had  about  four  or  five  people  he  was  watching  with  this  interval  sampling. 
It  turned  out  to  be  a  very  reliable  technique.  Later,  researchers  actually  checked  the  sampling 
technique  against  a  very  detailed  analysis  taken  with  a  motion  picture  camera.  Reliability  was  very 
high — up  in  the  .90s. 

With  this  activity  sampling  method  in  hand,  Chris  was  ready  to  examine  how  navigators  spent  their 
time  on  these  Arctic  missions.  He  started  with  some  control  missions,  for  a  baseline  condition.  The 
controls  were  mid-latitude  navigation.  Chris  flew  over  the  Atlantic  and  the  Pacific,  finding  out 
what  navigators  did  for  those  expanses.  Then  he  did  the  Arctic  missions.  He  was  looking  forward 
to  seeing  the  North  pole,  but  he  never  did.  When  the  airplane  he  was  on  flew  over  the  North  pole, 
it  was  so  cloudy,  he  couldn't  see  down  and  he  couldn't  see  the  sky,  although  he  did  see  it  on  the 
radar  scope. 

"In  the  Arctic,  the  navigators  didn't  have  a  decent  radio,  so  they  just  kind  of  felt  their  way  until 
they  could  see  the  stars  or  see  the  ground,  or  at  least  get  drift  readings.  You  get  incredible  wind 
shifts  in  the  Arctic,  that's  the  problem.  And  you  get  very  high  velocity  winds.  You  may  get  winds 
of  60,  70,  or  80  knots,  and  if  you're  wrong  on  the  direction,  you  can  end  up  a  long  way  away 
from  where  you  think  you  are." 

Chris  flew  out  of  Fairbanks,  Alaska  in  three  18-hour  missions,  and  then  he  flew  over  to  Goose 
Bay,  Labrador.  In  Goose  Bay  he  got  a  chance  to  say  hello  to  Charles  Lindbergh.  Lindbergh  was 
apparently  doing  some  flying  for  the  Air  Force  -  all  by  himself;  "no  crew  or  anything." 


Chris’s  philosophy  was  that  you  had  to  become  a  part  of  the  team,  do  the  job,  and  be  accepted  as 
pne  of  the  crew.  For  example,  during  a  pre-flight  inspection  he  would  stand  at  the  end  of  the  line 
of  crew  to  be  counted  as  one  of  them.  When  it  came  time  to  start  the  aircraft  up,  he  would  get  in 
line  to  help  turning  over  the  propellers.  The  biggest  compliment  he  could  get  was  to  be  asked  to 
go  out  for  a  beer  by  one  of  the  crew  after  a  mission.  This  philosophy  was  maintained  for  all  his 
projects,  even  when  he  didn't  know  the  job  himself,  for  example  working  down  on  the  factory 
floor. 
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One  trip  in  particular  stood  out.  They  were  up  for  16-18  hours  and  Chris  got  hungry  first,  opened 
his  boxed  lunch,  and  found  one  of  the  most  unlikely  things  you  can  think  of  to  make  a  sandwich 
out  of:  cold  pancakes.  He  had  cold  pancakes  between  two  slices  of  bread.  He  thought,  "well,  the 
cook  down  there  knows  I'm  a  civilian,  and  he's  just  trying  to  make  fun  of  me."  And  he  told  the 
pilot,  "Gee,  a  cold  pancake  sandwich!"  It  was  miserably  cold  on  the  airplane.  The  pilot  opened  his, 
and  he  too  had  a  cold  pancake  sandwich.  "When  the  airplane  hit  the  ground,  it  had  barely  stopped 
rolling  before  the  pilot  was  out  of  that  airplane  and  he  was  running  for  that  cook  shack.  He  was 
furious.  The  poor  cook  said,  look,  the  supplies  didn't  come  in  and  the  only  thing  I  could  think  to 
make  sandwiches  out  of  was  pancake  dough." 

Chris  worked  out  a  regime  of  resting  about  ten  minutes  out  of  the  hour.  He  had  to  last  for  16-18 
hours.  He  took  samples  every  five  seconds  for  50  minutes.  And  he  tried  to  vary  the  ten  minutes 
because  the  navigator's  work  is  quite  periodic.  If  he's  a  good  navigator,  every  15  minutes,  he 
might  read  drift,  and  every  30  minutes,  he  might  take  a  sun  shot.  That's  why  it  was  important  to 
stagger  the  ten  minutes. 

The  data  from  the  Arctic  missions  were  strikingly  different  from  what  he  found  at  the  lower 
latitudes.  Chris's  data  showed  what  the  additional  burden  was  on  a  navigation  team.  In  the  mid¬ 
latitudes,  one  navigator  was  doing  the  entire  job.  In  the  Arctic  regions,  they  had  teams  of  four  or 
five  trying  to  do  the  navigation.  If  they  had  had  a  decent  gyro,  one  or  two  guys  could  easily  do  it. 
One  of  the  penalties  for  the  extra  crew  members  was  that  every  crew  member  you  added  cost  you 
in  terms  of  fuel.  Just  adding  him  and  his  gear,  and  his  food,  and  everything  else,  including  all 
kinds  of  survival  gear  all  over  the  airplane,  added  up  to  a  hefty  penalty.  An  individual  crew 
member  at  that  time  was  worth  about  1200  lbs.  of  fuel. 

Chris  had  not  discovered  the  need  for  better  directional  equipment.  Anybody  who  ever  navigated  in 
the  Arctic  region  knew  that.  The  significance  of  his  findings  was  to  lend  substance  to  the  need.  He 
provided  some  quantitative  data  about  what  was  happening  in  Arctic  navigation. 

Back  at  Wright-Patterson  Chris  wrote  a  few  reports.  He  consulted  on  these  reports  with  friends  in 
the  old  Equipment  Laboratory  on  base.  They  were  responsible  for  developing  better  arctic 
navigation  equipment.  The  gyros  at  the  time  were  simply  inadequate.  They  might  have  precession 
rates  of  five  degrees  an  hour,  or  they  might  generate  a  precession  rate  of  forty  degrees  an  hour. 

The  navigator  had  no  way  of  telling.  They  all  agreed  that  with  better  gyroscopes,  navigators 
wouldn't  have  to  rely  on  their  magnetic  compasses. 


Eventually  the  Air  Force  did  put  sighificant  ainbunts  of  money  into  the  development  of 
gyrocompasses  primarily  with  Miiiheapblis  Honeywell.  The  end  result:  the  gyrocompasses  now 
are  so  accurate  that  you  can  fly  over  the  pole  or  any  where  in  the  world  and  the  precession  rates 
are  insignificant.  And  without  the  Air  Force's  work  on  gyroscopes,  the  space  program  would 
have  run  into  great  difficulties: 


Summary 

It's  hard  to  document  the  impact  of  studies  like  this.  The  Air  Force  may  have  gone  ahead  even 
without  the  data.  Possibly  the  data  Chris  collected  speeded  up  the  program,  but  no  one  can  say  by 
how  much.  In  an  organizational  setting  with  overlapping  teams,  there  is  rarely  a  chance  to  tease  out 
the  influence  of  any  single  strand  of  data. 
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In  this  project,  Chris  learned  about  the  value  of  field  data.  He  learned  the  importance  of  actually 
going  out  and  trying  to  experience  the  job  himself  or  interacting  very  closely  with  people  who  are 
doing  the  job,  a  lesson  he  carried  with  him  throughoU  his  career. 


He  also  learned  that  there  has  to  be  full  disclosure  of  what  the  researcher  is  there  for.  It  doesn't 
help  to  flash  letters  iii  front  of  them,  saying,  "General  so-and-so  said  I  could  come  up  here  and 
fly."  "It's  important  to  meet  with  the  people  being  studied,  tell  them  quite  candidly  why  you're 
there,  and  what  you  hope  to  do.  You're  not  there  to  check  on  them,  you're  there  to  check  on 
method,  and  so  on.  You're  not  there  to  try  to  denigrate  anyone's  activities,  reputation,  or  anything. 
You're  there  to  help  them,  really,  in  the  long  run,  you  hope  what  you  do  will  help  them." 

Chris  also  sent  to  the  teams  he  worked  with  a  copy  of  the  reports  he  wrote  as  a  result  of  being 
there.  They  seemed  to  appreciate  that.  The  reports  gave  full  credit  to  the  personnel  of  the  units,  and 
that  helped  them  understand  the  importance  of  their  contribution. 


A  further  lesson  was  the  importance  of  trying  to  set  up  something  consistent  with  fairly  decent 
experimental  methods  in  the  field.  That  Was  the  rationale  for  studying  the  ihiddatitude  flights,  as 
a  control  group.  You  can't  set  Up  perfect  experiments  in  the  fields  but  often  you  can  get  some 
kind  of  control  data,  some  basis  for  comparison.  In  this  case,  the  differences  between  the  two 
latitudes,  the  high-latitudes  and  the  mid-latitudes,  Were  just  Striking. 
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Going  Ballistic  for  Human  Factors 

Standards 

An  Interview  with  Donald  Topmiller 


How  does  human  factors  knowledge  get  incorporated  into  the  design  process?  How  does  the 
human  factors  engineer  add  clout  to  design  and  redesign  recommendations? 

These  are  some  of  the  problems  that  have  frustrated  HF  experts  for  years.  In  the  mid-fifties  Donald 
Topmiller  of  the  Aero  Medical  Research  Laboratory  was  facing  the  very  same  questions.  He  and 
his  colleagues  were  often  required  to  take  part  in  Developmental  Engineering  Inspections  and 
provide  design  recommendations.  However,  the  recommendations  they  made  carried  little  or  no 
weight;  they  had  no  official  documentation  to  back  them  up,  and  the  contractors  were  not  required 
to  follow  human  factors  guidelines  that  had  been  set  out  in  various  unofficial  publications.  The 
contractors  were  thus  not  responsible  for  the  incorporation  of  human  factors  recommendations. 

The  result  of  these  problems  was  poorly  human-factored  systems,  and  it  frustrated  HF  engineers  to 
know  that  they  had  many  recommendations  to  offer,  but  they  were  still  having  little  impact  on 
design. 

In  the  fall  of  1957  Lt  Col  Bob  Lacey,  a  former  Aero  Medical  lab  researcher,  invited  Topmiller  and 
his  three  office  mates  (Lt  "Kib"  Kibler,  Dave  Greek,  and  Charlie  Bates)  to  come  to  the  Ballistic 
Missile  Division's  TBST  facility  at  Cape  Canaveral.  He  wanted  them  to  take  a  look  at  some  of  the 
equipment  and  systems  being  implemented  at  the  Ballistic  Missile  Division  and  to  make  some 
informal  evaluations  of  these  new  systems.  With  the  increasing  intensity  in  the  cold  war,  the 
Ballistic  Missile  Division  had  been  introducing  many  new  systems  and  equipment  with  little  regard 
for  user  requirements. 


The  initial  catalyst  for  the  impact  of  this  team  was  the  invitation  from  a  former  AMRL  researcher. 
The  fact  that  Lt  Cbl  LaGey  had  worked  at  AMRL  meant  that  he  was  aware  of  the  \yprk  that  went 
on  there  and  of  the  potential  impact  of  that  work  on  his  new  assignment  to  Ballistic  Missile 
Division  and  the  problems  he  Was  seeing. 


As  an  example,  the  refueling  personnel  at  Canaveral  were  required  to  wear  protective  gloves  which 
physically  prohibited  them  from  doing  their  job  safely.  The  cumbersome  protective  gloves  meant 
that  they  were  unable  to  undo  the  fasteners  on  some  of  the  highly  flammable  and  toxic  liquid 
containers.  Operators  were  removing  their  gloves  in  order  to  do  the  job  and  in  doing  so  exposing 
themselves  to  great  hazards.  In  one  instance,  rocket  fuel  had  soaked  into  a  refueler's  undergarment 
and  was  ignited  in  the  locker  room  when  the  man  lit  a  cigarette. 

A  further  example,  discovered  by  Topmiller  and  his  associates,  was  the  fact  that  control  consoles 
for  a  particular  system  were  being  designed  by  six  different  subcontractors  none  of  whom  had  any 
regulations  guiding  the  human  factors  input  for  their  designs.  It  was  conceivable  that  the  six 
different  contractors  could  have  come  up  with  six  different  ways  to  accomplish  the  same  function. 
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Topmiller  and  his  colleagues  spent  over  six 
weeks  at  Cape  Canaveral.  They  soon  began  to 
wonder  at  the  inconsistency  in  the  application 
of  human  factors  knowledge  in  the  design  of 
many  critical  systems  and  important  equipment. 

At  the  end  of  each  day  the  group  would  swap 
"horror  stories"  from  their  day’s  evaluation.  They  began  to  talk  about  what  might  be  done  to 
improve  the  problems  at  Cape  Canaveral  as  well  as  improve  their  chances  of  having  an  impact  on 
system  development  at  an  early  stage.  They  were  tired  of  being  brought  in  when  too  much  had 
already  been  invested  in  the  design,  "after  the  tin  had  been  bent." 

On  their  return  to  the  Aerospace  Medical  lab  at  Wright-Patterson,  Topmiller  and  his  colleagues 
submitted  an  internal  report  on  their  trip  to  Cape  Canaveral  describing  what  they  had  found.  They 
also  began  to  throw  ideas  around  their  office  about  how  they  might  accomplish  the  fix. 


The  opportunity  to  bounce  ideas  off  one 
another  as  office-mates  and  during  the  trip  to 
Canaveral  fostered  teamwork,  brainstorming, 
and  communication  on  a  daily  basis. 


Part  of  the  "fix"  for  Cape  Canaveral's  problems  was  the  same  fix  needed  by  the  human  factors 
engineers  to  add  clout  to  their  design  recommendations:  mandate  certain  human  factors  practices 
and  standards,  and  require  designers  to  document  their  human  factoring  efforts  to  demonstrate 
how  these  efforts  complied  with  the  required  standards. 


Up  until  that  point  the  process  of  including  human  factors  in  design  had  relied  on  guidelines  like 
the  Joint  Services’  Human  Engineering  Guide  to  Equipment  Design.  However,  these  guidelines 
were  not  required  to  be  followed  and  provided  no  ammunition  for  the  human  factors  engineer  to 
use  with  the  designers  who  had  done  a  poor  job  of  considering  the  needs  of  the  user. 


Having  appreciated  the  existing  problem,  Topmiller  and  his  colleagues  decided  to  invest  some 
time  into  doing  something  about  it  rather  than  just  sitting  back  and  complaining  about  it. 


In  order  to  standardize  the  lab’s  evaluation  process,  Topmiller,  Kibler,  Bates,  and  Greek  put 
together  a  checklist  of  human  factors  considerations  for  design.  This  informal  checklist  allowed 
them  to  participate  in  an  evaluation  with  a  means  of  providing  standardized  feedback  to  the 
designer  based  on  their  checklist.  They  were  now  able  to  point  out  improvements  to  the  design  and 
point  to  the  evidence  to  support  their  suggestions. 

In  1958,  following  the  internal  checklist,  the  group  put  together  a  technical  document,  MIL-H- 
26207,  which  listed  requirements  for  contractors  to  provide  certain  types  of  information  recording 
the  incorporation  of  human  factors  guidelines  into  the  design  process.  These  requirements  included 
the  need  for  information  about  any  functions  analyses,  task  analyses,  and  anthropometric  analyses 
that  had  been  used  to  help  the  design.  If  a  design  failed  to  take  into  account  certain  types  of  human 
factors  information,  the  human  factors  engineer  now  had  a  document  to  point  to  support  the 
recommendations.  This  data  specification  also  encouraged  a  standardized  approach  to  the 
incorporation  of  human  factors  guidelines  into  design. 

The  team  continued  to  pursue  the  idea  of  regulation  and  standardization  by  producing  the  first 
military  standard  document  which  provided  human  factors  standards  and  requirements  for  design, 
at  the  request  of  Lt  Col  Lacey.  MIL-STD-803  laid  out  human  factors  standards  to  be  achieved  by 
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designers  of  ballistic  missile  systems.  This  was  soon  followed  by  a  similar  document  for  aircraft 
systems  design  and  ground  equipment  design  (see,  The  Case  of  the  Uncommon  Consoles:  An 
Interview  with  Steve  Heckart). 


One  of  the  surprising  aspects  of  the  story  is  the  time  scale.  Within  a  period  of  less  than  three 
years,  Topmiller  and  associates  had  produced  and  published  standards  for  the  Ballistic  Missile 
Division.  This  rapid  time  course  is  unlikely  to  occur  that  quickly  today. 


These  were  the  first  steps  to  requiring  systems  designers  to  think  about  the  operators  of  their 
systems  and  how  they  might  design  more  effectively  for  the  user.  Not  only  that,  but  these 
standards  provided  the  means  by  which  the  HF  engineer  could  evaluate  designs  based  on  accepted 
standards,  and  provide  redesign  recommendations  with  documented  support.  These  formalized 
human  factors  specifications  and  standards  launched  the  field  into  an  era  of  acceptance  by  the 
engineering  community  in  government  as  well  as  in  industry. 


The  Case  of  the  Uncommon  Consoles 

An  Interview  with  Steve  Heckart 


The  balance  between  centralization  and  decentralization  is  difficult  at  all  levels  of  organizations.  If 
you  centralize  too  much,  you  need  all  sorts  of  special  exceptions  and  these  keep  growing  until  the 
system  bogs  down.  If  you  centralize  too  little,  you  have  waste  and  duplication  and  inefficiency. 
There  are  no  simple  answers,  and  the  only  guarantee  is  that  the  issue  will  be  a  constant  challenge. 

One  way  that  the  issue  of  centralization  arises  in  the  field  of  human  factors  is  over  standards.  In  the 
previous  case  we  found  Don  Topmiller  trying  to  impose  standards  for  the  Ballistic  Missile 
Division.  This  case  describes  how  Steve  Heckart  took  those  standards  a  step  further,  to  form  MIL- 
STD-1472,  the  central  directive  for  the  use  of  human  factors  in  design.  Military  standards  are  now 
in  place  for  the  combined  services;  Steve  Heckart  served  as  the  focal  point  for  Air  Force  inputs  into 
the  standards. 

Prior  to  1958,  there  were  no  human  factors  standards  for  designers  and  contractors  to  adhere  to.  In 
the  late  1950s  Aero  Medical  Research  lab  was  instmmental  in  getting  the  first  standards  mandated, 
requiring  contractors  to  provide  evidence  of  some  consideration  of  the  human  operator  in  systems 
design  (see  Going  Ballistic  for  Human  Factors  Standards:  An  Interview  with  Don  Topmiller). 

Heckart's  particular  interest  was  in  standards  for  the  design  of  the  ground-based  equipment  and 
systems,  maintenance  equipment,  and  contrbl  panel  design.  When  Lt  Col  Bob  Lacey  had  invited 
the  Aero  Medical  laboratory  people  out  to  Ballistic  Missile  Division  at  Cape  Canaveral  to  talk  about 
developing  a  human  engineering  design  standard,  Heckart  and  his  co-workers  had  also  been 
approached  by  Capt  Ed  Milauckas  on  the  same  topic.  Milauckas  was  concerned  about  the  design  of 
missile  complexes,  launch  officer  consoles,  and  silo  consoles  and  equipment. 

Each  specific  system  had  its  own  custom-built 
console.  The  contractors  loved  it  because  they 
had  a  brand  new  design  problem  for  each  new 
project.  Milauckas  hated  it  because  the  Air  Force 
wound  up  spending  large  sums  of  money  each 
time,  and  because  the  differences  between 
consoles  made  the  operators’  jobs  more  difficult, 
and  made  it  harder  to  maintain  all  the  consoles. 

The  differences  seemed  trivial  and  arbitrary  to 
Milauckas  and  he  felt  that  it  should  be  possible 
to  develop  a  common  format  for  everyone  to  use. 

Milauckas  and  Heckart  discussed  the  idea  that  if  a  standard  set  of  control  panels  could  be  designed 
according  to  human  factors  design  criteria,  the  cost  of  producing  these  standard  panels  would  be 
less  than  having  custom  built  panels  for  each  application.  Not  only  that  but  the  equipment  would  all 
be  standardized  and  designed  according  to  the  human  factors  standards.  The  panels  could  be  made 
so  that  they  could  be  adapted  for  special  purpose  systems,  but  the  added  cost  of  adaptation  would 
be  less  than  having  the  contractor  create  new  designs  for  each  system. 

Charlie  Bates,  Ken  Kennedy,  Milt  Alexander  and  Heckart  kicked  around  the  question  of  whether 
standard  control  panel  design  could  be  accomplished,  and  whether  expensive  special  purpose 


Heckart's  goal  was  that  all  the  lab's  research 
should  have  some  application  to  the  mission 
of  the  Air  Force,  and  not  just  for  the  purposes 
of  scientific  publications.  One  way  of 
achieving  this  was  to  provide  data  which 
could  be  applied  as  design  criteria  in 
standards. 
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designs  were  needed  or  could  be  adapted  from  the  standard  panels.  Up  to  that  time,  human  factors 
and  ergonomics  had  primarily  addressed  hardware  issues  such  as  the  shape  of  knobs  and  buttons, 
the  motions  and  sensitivity  to  operate  tools  while  wearing  gloves,  the  reach  distance  for 
instruments  in  a  cockpit,  and  so  forth.  Applying  human  factors  standards  to  the  logic  of  console 
design  was  something  new.  Yet  it  seemed  feasible.  The  team  agreed  that  they  should  take  a  shot  at 
it. 


Heckart's  criterion  for  evaluating  whether  he  had  done  a  good  job  was  to  see  how  inany  pieces  of 
tape,  grease  pencil  marks,  job  aids,  and  other  work  arounds  had  been  applied  to  a  piece  of 
equipment.  If  there  were  too  many  then  the  designers  had  done  a  poor  job  and  the  design 
standards  had  failed,  or  had  not  been  applied; 


Heckart  knew  that  the  data  for  these  purposes  already  existed,  it  was  just  a  matter  of  finding  them, 
applying  them  to  the  control  panels,  and  testing  their  suitability  for  the  tasks  that  the  operators 
would  have  to  accomplish  with  them. 

Heckart  and  Ken  Kennedy,  the  reach  data  specialist,  began  to  draw  the  relevant  data  together. 
Heckart  observed  the  operators  who  would  be  using  the  panels  in  order  to  visualize  how  the 
control  panels  were  being  used.  Kennedy  built  mock-ups  of  the  panels  in  his  shop.  After 
evaluating  the  mock  ups,  the  standards  for  several  control  panel  designs  were  put  into  a  draft 
standard  and  eventually  were  implemented  in  the  military  standards  document,  MIL-STD-803 A. 
The  time  course  of  the  whole  project  was  only  18  months  from  Milauckas's  problem  statement  to 
the  first  version  of  MIL-STD-803 A. 


Heckart  believes  that  the  human  factors  standard?  requir^^  an  advocate  in  the 
coUld  make  those  tailoiihg  decisions  and  apply  them  to  the  design  contract  1 
administratively  fesponSiMe  to  some  external  organization,  such  as  the  Aero  1 

design  office  who 
bis  person  should  be 
Vledieal  lab,  and  riot 

to  the  design  engineers  in  the  SPO.  The  Arrriy  did  a  better  job  than  the  Air  Fc 

)rce  in  this  respect,  as 

their  human  factors  people  Were  spread  throughout  the  arsenals  and  were  in  a 

1  better  position  to 

advocate  the  inclusion  of  design  standards  in  contractors'  design^specificatioi 
the  appropriate  tailonng  of  standard  design  criteria  to  fit  the  specific  applicati( 

IS  and  also  to  ensure 
on. 

One  of  the  recurrent  problems  is  that  once  standards  have  been  provided  to  the  design  cominunity 
they  don’t  get  implemented  into  design  contract  requirements.  One  of  the  reasons  they  aren't  put 
into  the  design  contracts  is  that  the  standards  are  so  inclusive  that  some  sections  may  not  seem  to 
apply  to  the  particular  design.  The  primary  purpose  of  the  standard  control  panel  designs  was  to  be 
adaptable  for  tailoring  to  the  needs  of  specific  users.  Second  was  the  goal  of  cutting  production 
costs  of  custom  control  panel  designs  for  every  new  project.  The  Army  has  made  even  greater  use 
of  MIL-STD-1472  than  the  Air  Force,  because  the  Army  provides  advocates  and  interpreters  of  the 
standards  in  the  design  offices. 
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"Measure  Me  Ulorkload! " 


An  Interview  with  Bob  O'Donnell 


In  1977  Charlie  Bates  made  Dr.  Robert  O'Donnell  head  of  a  new  branch  and  told  him  to  go  out  and 
"Measure  me  workload!"  The  Workload  and  Ergonomics  branch  was  thus  bom.  O'Donnell  set 
about  building  the  branch  and  establishing  a  reputation  for  its  work.  At  the  time  workload  was  not 
the  buzzword  that  it  later  became.  In  that  sense,  Armstrong  lab  was  leading  the  way  again. 

The  fact  that  no  one  knew  what  workload  really  was  did  not  deter  O'Donnell  from  accepting  the 
challenge,  but  O'Donnell  is  now  the  first  to  admit  that  he  is  still  unable  to  measure  the  elusive 
concept  of  workload.  Yet,  the  research  that  came  from  that  branch  under  O'Donnell's  leadership 
has  produced  some  ground  breaking  work  in  the  measurement  of  human  performance  and  in  the 
assessment  of  operator  readiness  for  duty. 

O’Donnell  knew  the  research  approach  that  he  was  most  interested  in  pursuing.  And  he  also  knew 
the  research  approach  that  people  thought  would  have  the  highest  likelihood  of  success.  But  given 
the  importance  he  placed  on  needing  a  rigorous  and  balanced  program,  he  added  a  third  approach 
that  was  neither  interesting  to  him  nor  likely  to  contribute  much  of  use.  And  it  is  that  third 
approach,  the  ugly  duckling  of  the  program,  that  has  had  the  greatest  impact. 

An  Integrated  Approach 

With  the  challenge  issued  by  Bates,  O'Donnell  set  about  organizing  his  branch.  His  first  concern 
was  to  ascertain  what  it  meant  to  measure  workload.  There  was  not  a  wealth  of  literature  out  there 
addressing  the  issue.  The  literature  that  did  exist  could  not  provide  a  succinct  definition  that  would 
enable  O'Donnell  to  produce  an  instmment  that  would  give  him  a  single  reading  of  workload. 

O'Donnell  made  his  first  key  decision:  he  was 
not  going  to  find  workload  in  any  one  place  so 
he  was  going  to  need  an  integrated  approach. 

He  divided  the  branch  into  three  specialty 
areas:  physiological  assessment,  subjective 
reporting  techniques,  and  behavioral  and 
cognitive  performance  assessment.  He  added 
in  this  third  component  because  it  seemed  to 
balance  the  program.  He  didn't  have  any 

expectation  that  the  work  on  the  behavioral/cognitive  measures  would  be  successful,  but  he 
thought  it  would  help  to  focus  the  other  two  approaches.  Nevertheless,  to  make  sure  his  biases 
didn't  get  in  the  way,  O’Donnell  treated  each  approach  as  equally  valid  and  worthy  of  equal 
attention,  until  one  proved  its  worth  over  the  others. 

In  order  to  develop  these  three  approaches  for  an  integrated  view,  O'Donnell  knew  he'd  need  an 
eclectic  group  of  researchers  who  were  self-directed  and  focused.  His  next  task  was  to  identify 
those  individuals  and  bring  them  on  board. 

Clark  Shingledecker,  an  NIH  Fellow  and  SRL  employee,  was  taken  on  to  handle  the  behavioral 
assessment  group,  working  with  Mark  Crabtree.  Gary  Reid,  who  was  working  in  personnel 


O'Donnell's  own  background  was  in 
physiological  performance  assessment;  and  he 
could  have  been  tempted  to  stick  with  what  he 
knew  and  put  all  his  money  on  one  horse,  the 
physiological  measures.  Instead  he  chose  a 
balanced  approach  bringing  in  experts  from  other 
disciplines,  and  favored  no  one  line  of  research. 
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testing  at  Williams  AFB  at  the  time,  was  recmited  to  handle  the  subjective  workload  assessment 
issues.  The  third  group,  the  physiological  group,  was  headed  up  by  Glenn  Wilson  after  a  chance 
meeting  with  O'Donnell  when  Wilson  brought  his  class  from  Wittenberg  University  to  look  round 
the  laboratories.  A  fourth  figure  was  brought  on  as  an  information  resource — ^Tom  Eggemeier  from 
Wright  State  University  (now  the  Chair  of  the  Psychology  Department  at  the  University  of 
Dayton). 

Playing  the  Circuit 

Having  mustered  his  troops,  O'Donnell  let  them  do  what  they  did  best.  His  only  mandate  was  that 
the  group  should  be  a  high-profile  group  in  meetings  and  conferences,  and  that,  even  though  they 
were  working  in  their  own  groups,  they  would  always  try  to  present  their  work  in  integrated 
forums.  The  workload  branch  became  known  for  stacking  a  session  and  presenting  their  work  as  a 
team.  O’Donnell  would  introduce  the  integrated  approach,  Eggemeier  would  provide  the 
background  information  from  the  literature,  Reid  would  go  over  the  subjective  work, 

Shingledecker  the  performance  work,  and  Wilson  the  physiological  work.  The  well-greased 
workload  presentation  machine  became  a  high  visibility  laboratory  with  a  respected  reputation. 


O'Donnell  intentionally  pushed  the  high  profile  of  the  branch,  the  purpose  being  to  invite 

criticism.  The  workload  group  was  as  much  searching  in  the  dark  for  the  elusive  concept  as  all  the 
other  researchers  interested  in  workload.  By  taking  a  spotlight,  their  integrated  approach  was  open 
to  feedback,  and  not  all  of  it  was  supportive  of  their  work. 


The  Ugly  Duckling 

Shingledecker's  work  in  the  lab  had  produced  a  cognitive  performance  test  battery  called  the 
Criterion  Test  Set  (CTS).  Examples  of  the  tests  used  include  tracking  tasks,  mental  rotation, 
continuous  memory,  semantic  reasoning,  and  the  Sternberg  reaction  time  task.  Shingledecker  had 
originally  seen  the  purpose  of  the  test  battery  being  to  provide  performance  criteria  against  which 
other  measures  of  workload  could  be  tested. 

However,  the  CTS  was  never  actually  used  as  it  had  been  intended.  Instead,  people  realized  that  it 
could  be  used  to  measure  how  much  environmental  stressors  degraded  cognitive  performance.  If 
we  expect  pilots  to  function  well  under  extreme  conditions,  we  should  at  least  try  to  find  out  what 
these  conditions  do  to  the  ability  to  think  clearly.  The  CTS  allowed  such  studies  to  be  done. 
Different  degrees  of  a  stressor  (e.g.,  thermal  or  chemical)  were  applied  while  the  subjects  took  the 
CTS  tests.  The  scores  from  the  CTS  were  compared  to  assess  how  performance  was  degraded  by 
the  stressor.  This  unintended  use  meant  that  the  CTS  battery  could  provide  answers  about  how 
stressors  affect  performance. 

The  next  step  occurred  in  1980,  when  Fred  Hegge  of  the  Army  approached  ODonnell  who  was 
part  of  a  Tri-Service  commission  on  counteracting  chemical  weapons.  The  military  had  identified  a 
number  of  agents  that  worked  to  counteract  the  effect  of  chemical  weapons,  but  no  one  was  sure 
how  these  agents  would  affect  the  pilots  themselves,  in  terms  of  the  cognitive  and  behavioral 
performance.  Would  they  be  less  likely  to  make  discriminations,  or  form  judgments,  or  respond 
quickly?  The  Air  Force  was  more  advanced  in  the  area  of  stressor  effects  on  performance  than  the 
other  services,  so  O'Donnell  was  given  the  task  of  developing  a  technique  to  assess  the  cognitive 
performance  effects  caused  by  these  agents. 
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O'Donnell's  group  adapted  the  existing  CTS  which  had  already  proven  helpful  in  the  area  of 
assessing  stressor  effects.  In  1983,  they  produced  a  new  cognitive  assessment  battery  which 
utilized  about  60%  of  the  tests  from  the  CTS,  and  added  in  several  other  accepted  and  established 
cognitive  performance  tests  to  make  up  the  complete  test  battery. 

This  cognitive  assessment  battery  is  still  being  used  today,  and  has  also  been  adapted  by  NATO  as 
a  standard  performance  test  battery  for  cognitive  performance  assessment. 

O'Donnell  retired  from  the  Armstrong  Laboratory  in  1985.  Since  then  he  has  further  modified  the 
cognitive  assessment  battery  to  provide  an  easy  to  apply,  easy  to  use,  cognitive  performance 
assessment  tool  for  workplaces  demanding  high  levels  of  alertness  and  accurate  judgments. 
Workers  take  a  3-6  minute  test  before  they  go  on  their  shift.  If  they  fail  to  reach  a  criterion  level 
based  on  their  baseline  scores,  they  are  considered  unfit  to  perform  their  shift.  This  test  battery  has 
been  backed  by  the  FAA,  Amtrak,  Metro  Machine  Shipping  Company,  and  the  New  York  Metro 
as  a  quick  assessment  of  mental  alertness  before  a  working  shift. 


Although  O'Donnell  was  unable  tb  satisfactorily  grant  Bates'  original  request,  much  has  been 
accomplished  by  the  branch.  The  researchers  have  made  important  findings  in  the  area  of 
physiological  measurement  techniques,  and  the  development  of  ah  effective  subjective  workload 
assessment  technique  (SWAT);  What  was  not  expected  was  that  the  most  successfiil  applications 
have  come  from  the  ugly  duckling.  Cognitive  peiformance  assessment  teehriiques. 
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Moments  of  Inertia 

An  Interview  with  Charlie  Clauser 


How  much  does  your  head  weigh?  How  does  the  center  of  mass  of  your  body  shift  when  you 
move  your  limbs?  You  may  not  know  the  answers  to  these  questions,  but  Charlie  Clauser  does. 
This  is  a  story  of  baboons  and  body  parts  and  an  unplanned  journey  that  resulted  in  a  set  of  data 
from  1974  that  is  still  being  used  to  produce  manikins  for  crash  testing,  restraint  testing,  ejection 
seat  testing,  and  similar  equipment  design  issues.  The  data  are  even  being  used  to  explore  human 
performance,  such  as  in  gymnastics,  using  computer  modelling  techniques.  But  it  all  happened  by 
accident. 

The  Field  of  Physical  Anthropology 

Physical  anthropologists  trace  the  size  and  shape  of  human  structures  such  as  bones  and  skulls. 
Academic  physical  anthropologists  study  basic  issues  concerning  the  origin  of  the  species,  the 
history  of  man.  Applied  physical  anthropologists  try  to  use  data  describing  the  human  form,  its 
dimensions,  characteristics  and  properties,  in  order  to  model  human  and  human-machine 
performance. 

When  Charlie  Clauser  completed  graduate  school  at  Indiana  University  in  Anthropology  in  1956, 
he  joined  the  applied  camp,  going  to  work  at  the  Aero  Medical  Research  Laboratory  at  WPAro.  At 
that  time,  the  Anthropology  group  was  investigating  the  relevant  dimensions  and  characteristics  of 
the  human  body  in  order  to  help  with  cockpit  design,  ejection  seat  design,  and  the  design  of  other 
close  fitting  personal  protective  equipment.  (Today  this  group  is  called  the  Computerized 
Anthropometric  Research  and  Design  Group  of  the  Design  Technology  Branch.  The  term 
“anthropology”  reminded  too  many  people  of  Margaret  Mead  in  Samoa,  and  was  dropped.) 

Free-Floating  Curiosity 

In  the  late  1950s,  the  Air  Force  began  to  get  heavily  involved  in  the  space  program.  NASA  had  not 
yet  been  formed.  One  topic  of  study  was  the  problem  of  extra-vehicular  activities  (EVA). 
Astronauts  were  going  to  have  to  use  self-maneuvering  units  to  perform  tasks  in 
zero-G  environments  outside  the  spacecraft.  As  a  result,  the  anthropometry  group  at  AMRL  had 
begun  to  wonder  how  movements  of  the  human  would  affect  the  stability  of  the  maneuvering 
system.  In  order  to  build  a  predictive  model,  the  scientists  needed  to  understand  the  mass 
distribution  characteristics  of  parts  of  the  body.  Once  the  anthropologists  at  AMRL  obtained  those 
data,  they  could  help  system  designers  predict  the  flight  characteristics  of  ^tronauts.  If  the 
Astronaut  Maneuvering  Systems  was  discovered  to  be  unsafe  or  unstable  in  any  way,  they  could 
redesign  it  based  on  the  inertial  properties  of  the  astronaut  from  the  work  of  AMRL 
anthropologists. 

Soon,  data  sponsored  by  AMRL  began  to  appear.  The  North  American  study  of  mass  and  centers 
of  mass  of  the  living  human  body  was  published  in  1963.  In  1964  Hanavan  published  the  first 
model  of  the  inertial  properties  of  the  full  human  body  based  on  these  data.  Clauser  had  initiated 
and  monitored  these  projects,  but  he  was  still  curious  about  how  the  data  on  mass,  and  center  of 
mass,  would  affect  Extra  Vehicular  Activity.  During  this  time,  Clauser’ s  desk  was  often  strewn 
with  wooden  "limbs,"  which  he  used  to  try  to  simulate  the  body’s  reactions  to  various  movements 
imposed  by  a  vehicle  in  space. 
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A  Lucky  Encounter 


Clauser  had  been  collaborating  with  a  group  from  CAMI  (Civil  Aero  Medical  Institute)  run  by  the 
FAA  in  Oklahoma  City.  They  had  conducted  a  cadaver  study,  sponsored  by  NASA,  in  order  to 
characterize  the  mass,  location  of  the  center  of  mass,  and  volume  of  the  human  body.  This  had 
been  published  in  an  AMRL  tech  report  in  1969. 

In  1972  Clauser  attended  a  meeting  on  the  west  coast  and  prior  to  returning  to  Ohio  he  arranged  to 
visit  an  old  graduate  school  friend,  Joseph  Young,  with  whom  he'd  worked  at  CAMI  on  the  first 
cadaver  study.  While  he  was  there,  Dick  Chandler,  who  did  research  at  CAMI,  had  seen  some  of 
the  work  that  had  come  out  of  AMRL  and  the  first  study,  and  took  the  opportunity  to  talk  with 
Clauser.  Thus  far,  AMRL  had  concentrated  on  studying  mass,  center  of  mass  location,  and 
volume.  Chandler  wanted  to  know  the  moments  of  inertia  of  the  body  segments. 

Clauser’ s  response  was  “What’s  a  moment  of  inertia?”  Not  being  an  engineer,  he  wasn't  familiar 
with  the  idea  but  he  quickly  realized  its  importance  once  Chandler  had  explained  what  he  meant. 

He  realized  that  the  stability  of  a  small  space  sled  would  be  governed  by  the  inertial  characteristics 
of  the  man/machine  system  and  that  these  characteristics  would  change  with  movements  of  the 
astronaut’s  arms  and  legs  and  head.  The  mathematics  and  physics  of  these  body  movements  were 
the  key  to  the  puzzles  that  he  had  been  struggling  with  in  order  to  understand  how  a  body  would 
move  in  a  zero-G  environment. 

Equally  important.  Chandler  knew  of  a  way  to  study  these  inertial  forces.  One  of  his  researchers  at 
CAMI,  Dr.  Herbert  "Mack"  Reynolds,  had  just  completed  such  a  study  with  baboons.  Therefore, 
the  techniques  for  making  the  necessary  measurements  had  already  been  developed.  Unfortunately, 
CAMI  did  not  have  the  manpower  to  undertake  such  a  project  for  the  human  body.  Chandler  asked 
Clauser  and  his  colleagues  if  they  would  be  interested  in  conducting  the  research  using  CAMI’s 
facilities  and  AMRL’s  manpower.  CAMI  also  had  another  important  resource,  which  was  their 
very  well-equipped  anatomy  laboratory;  CAMI  also  had  access  to  one  of  the  best  cadaver 
populations  in  the  country. 


Coincidental  circumstances  led  to  the  formation  of  this  collaborative  project;  Clauser  dropping  in 
on  his  friend;  Chandler  offering  Clauser  an  Opportunity  to  work  again  in  a  welL  equipped  lab  on  a 
mutually  interesting  topic,  the  inertial  properties  of  the  human  body;  and  Reynolds  having 
experience  in  the  irieasurement  techniques  of  these  properties  from  his  baboon  study. 


Cross-Country  Collaboration 

Clauser  returned  to  Dayton  and  wrote  up  a  short  proposal  to  initiate  money  from  the  National 
Highway  Traffic  Safety  Administration  (NHTSA).  He  proposed  the  research  that  had  been 
discussed  at  CAMI  and  tied  it  in  to  applications  to  transport  safety  and  the  self-maneuvering  units 
being  developed  by  NASA  for  EVA.  The  data  from  this  line  of  research  would  also  have 
applications  in  the  design  of  seat  restraints  in  vehicles  and  in  the  design  of  the  crash  test  dummies 
used  to  evaluate  vehicle  safety. 


The  applied  anthropology  community  fostered  collaboration  and  cooperation  between  researchers 
and  research  institutions.  This  enabled  Clauser  to  conduct  work  using  the  manpower  of  the 
Armstrong  Laboratory,  in  a  Well-equipped  anatomy  laboratory  owned  by  the  FAA,  with 
researchers  with  various  areas  of  knowledge  from  the  field  of  applied  anthropology,  and  the 
combined  funding  from  several  government  agencies:  NAS  A,  NHTSA,  FAA,  and  AMRL. 
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The  funding  was  approved  and  Clauser  had  the  go-ahead  for  the  study.  In  1973  Clauser, 

Reynolds,  Young,  and  Chandler  began  the  somewhat  macabre  task  of  measuring  the  moments  of 
inertia  of  the  body  parts  of  six  cadavers  in  the  CAMI  anatomy  lab  in  Oklahoma  City.  The  research 
project  had  been  suggested,  formally  proposed,  funded,  and  started  within  the  space  of  one  year,  a 
remarkable  achievement  in  itself.  The  measurement  procedure  itself  included  encasing  each  body 
part  in  a  box,  attaching  it  to  a  pendulum  and  setting  the  apparatus  in  motion.  Measurements  were 
taken  for  the  weight,  center  of  mass  location,  and  principle  moments  of  inertia  of  the  six  cadavers. 
The  segments  were  also  measured  individually  for  mass,  center  of  mass,  moments  of  inertia,  and 
volume. 


One  of  the  aspects  of  this  opportunity  presented  to  Clauser  and  his  Colleagues  was  the  fact  that  he 
was  allowed  to  pursue  the  project  in  the  first  place.  Clauser  commented  that,  in  those  days,  a 
researcher  worked  in  an  environment  which  encouraged  the  pursuit  of  questions  that  he  dr  she 
found  interesting,  even  if  the  original  idea  did  not  come  directly  from  the  lab,  or  if  the  work  itself 
was  not  to  be  conducted  in  the  lab.  The  positive  environment  in  the  Armstrong  lab  supported  and 
encouraged  scientific  curiosity  and  creativity  in  research  related  to  the  lab's  mission. 


The  data  collected  in  this  study  were  reported  in  an  AMRL  technical  report  (AMRL-TR-74-137, 
Chandler,  Clauser,  McConville,  Reynolds,  &  Young,  1975).  The  data  are  still  used  today  as  the 
standard  for  modelling  mass  distribution  characteristics  including  moments  of  inertia  for  the  human 
body.  Ironically,  the  most  immediate  use  that  the  Air  Force  made  of  these  data  was  for  designing 
ejection  seats,  rather  than  for  Extra  Vehicular  Activities.  In  order  to  predict  the  ejection 
characteristics  of  the  pilot  in  the  ejection  seat,  design  engineers  need  to  know  not  only  the  weight 
distribution  and  “flight”  characteristics  of  the  seat,  but  also  those  of  the  pilot.  The  data  collected  by 
Chandler  et  al.  from  both  the  CAMI  cadaver  studies  directly  addressed  this  issue. 


The  work  from  these  types  of  studies  tends  to  last  for  significant  periods  of  time.  Some  of  the 
earliest  anthropometric  data  gathered  in  the  1 800s  are  still  referred  to.  Clauser  and  colleagues  d^^ta 
from  these  studies  are  maintained  as  the  standard  Of  the  day  and  may  well  be  so  for  years  to  come, 
providing  accurate  data  for  manikins  and  performance  modeling. 


General  Reference 

Robinette,  K.  M.,  &  Fowler,  J.  (1988).  An  annotated  bibliography  of  United  States  Air  Force 
engineering  anthropometry  - 1946  to  1988.  AAMRL-TR-88-013.  Wright-Patterson  Air  Force 
Base,  OH. 
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Crossing  the  Barrier:  Deueioping  SAINT  to 
Modei  Human  and  Equipment  Dynamics 

An  Interview  with  Jerry  Chubb 


Design  engineers  rely  on  simulations  to  calculate  a  variety  of  factors  and  play  what-if  games  to 
examine  different  contingencies.  But  in  the  early  1970s,  design  engineers  did  not  have  a  way  to 
include  human  performance  issues  in  their  simulations.  In  fact,  they  were  suspicious  of  human 
performance  data  and  uninterested  in  the  attempts  to  model  these  data.  There  was  an  invisible 
barrier  between  the  design  engineers  and  the  human  factors  community.  Jerry  Chubb  was  in 
charge  of  a  project  to  help  design  engineers  use  human  performance  data.  He  had  to  find  a  way 
through  this  barrier.  The  software  simulation  he  and  his  team  produced,  SAINT  (Systems  Analysis 
of  Integrated  Networks  of  Tasks),  accomplished  the  trick.  It  took  about  six  years  to  develop,  from 
1970  to  1976. 

Jerry  started  working  for  the  Human  Engineering  Division  in  1963.  He  had  a  M.S.  in  psychology 
from  Ohio  State  University.  (He  later  went  on  to  get  his  Ph.D.)  He  inherited  this  project  without 
much  warning.  The  original  stimulus  for  the  effort  was  the  concern  in  the  late  1960s  about  how 
nuclear  attacks  would  affect  weapons  systems.  The  Special  Weapons  Center  at  Albuquerque,  NM 
commissioned  Vulnerability/Survivability  studies  on  airplanes  such  as  the  F-106,  the  B-52,  and 
others  to  see  if  the  equipment  would  survive  various  weapons  effects — the  thermal  pulse,  the  burst 
effect,  transient  electronics  effects,  electromagnetic  pulses,  and  other  ionizing  radiation  impacts. 
But  there  was  one  component  of  the  weapons  system  that  had  not  been  well  studied — ^the  pilot. 
Animal  research  had  been  going  on,  but  it  was  hard  to  extrapolate  from  these  experiments  to  the 
psychomotor  and  cognitive  processes  of  pilots.  This  gap  was  worrisome.  Contractors  studying 
this  problem  had  simply  cited  the  results  of  radiobiology  research  studies.  Little  was  said  about 
possible  performance  impacts. 

As  a  result,  a  meeting  was  held  between  representatives  of  the  Air  Force  Weapons  Laboratory  and 
Don  Topmiller  and  Bob  O’Donnell  of  the  Aerospace  Medical  Research  Laboratory.  Topmiller  and 
O’Donnell  suggested  that  the  problem  could  be  solved  using  modelling  techniques  such  as  the 
Siegel-Wolf  method.  This  was  a  simulation  model  that  had  been  developed  by  the  Navy  in  the 
early  1960s  to  use  task  analysis  data  to  estimate  task  times,  and  had  evolved  into  a  technique  for 
comparing  activity  timing  patterns  under  different  conditions.  You  could  use  the  Siegel-Wolf 
simulation  to  compare  task  timing  patterns  before  and  after  ionizing  radiation,  in  order  to  estimate 
the  magnitude  of  the  performance  decrement.  Therefore,  the  solution  seemed  close  at  hand.  As  a 
result,  the  Air  Force  Weapons  Laboratory  agreed  to  fund  the  project.  The  Human  Engineering 
Division  never  put  any  funds  into  the  project. 

Topmiller  and  O’Donnell  were  pleased  to  have  closed  the  deal,  and  considered  their  job  done.  Jerry 
Chubb  was  told  that  he  was  in  charge  of  the  new  project.  This  was  in  1970. 

Chubb  quickly  found  that  the  task  was  not  as  simple  as  he  had  been  led  to  believe.  He  began  by 
arranging  to  have  the  Siegel-Wolf  technique  modified  so  he  could  use  it  for  the 
Vulnerability/Survivability  program  for  the  F-106,  an  interceptor  aircraft.  When  Siegel  did  a 
literature  review  of  the  radiobiology  experiments,  he  discovered  lots  of  holes  that  limited  the 
generalizability  of  the  data.  If  they  wanted  to  estimate  human  performance  degradation,  it  was  clear 
that  they  would  need  to  rely  on  interpretations  of  the  radiobiology  data  by  Subject-Matter  Experts, 
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radiobiologists,  to  generate  subjective  estimates  of  how  behavioral  and  cognitive  performance 
would  degrade  as  a  function  of  different  radiation  dosages  and  at  different  times  following 
exposure. 

The  subjective  estimates  of  performance  degradation  were  based  on  animal  studies  and  case  studies 
of  humans  exposed  to  radiation.  And  to  get  these  subjective  estimates,  he  needed  to  derive 
cognitive  scales.  He  used  Guilford’s  Structure  of  Intellect  model,  selecting  three  of  the  five  scales 
(divergent  thinking,  convergent  thinking,  and  cognitive/perceptual  processing).  He  added  in  some 
psychomotor  factors  that  would  be  relevant  to  flying.  The  subjective  estimates  provided  him  with  a 
set  of  curves  showing  how  for  different  dosages  the  performance  degradation  would  increase 
sharply  following  exposure,  and  then  the  pilot  would  gradually  recover.  He  linked  these  curves  to 
other  estimations  of  the  probability  of  success  for  a  mission.  In  order  to  make  sense  of  all  these 
data,  a  simulation  model  was  needed.  The  simulation  served  as  a  bookkeeper  to  keep  all  the 
branching  and  permutations  straight. 

In  1971  Jerry  led  a  research  contract  to  extend  the  work  to  the  B-52,  which  had  a  larger  crew  than 
the  F-106.  This  contract  developed  a  model  that  could  handle  a  ten-person  crew.  The  model  also 
incorporated  other  psychological  constructs  that  would  be  important  for  describing  the  performance 
of  multi-person  crews. 

In  1972,  Jerry  went  shopping.  He  had  found  that  engineers  he  discussed  the  project  with  were 
reacting  negatively  to  the  Siegel-Wolf  technique.  As  far  as  he  could  tell,  the  negativity  was  due  to  a 
few  important  issues: 

•  The  Siegel-Wolf  technique  had  been  developed  by  behavioral  scientists  for  other  behavioral 
scientists,  so  engineers  didn’t  feel  that  it  was  speaking  to  them  in  their  language. 

•  The  Siegel-Wolf  technique  was  based  on  psychological  data  rather  than  the  hard  data  engineers 
preferred. 

•  Chubb’s  version  of  the  technique  was  based  on  Subject-Matter  Expert  estimates,  not  even 
empirical  data. 

•  Chubb  was  relying  on  the  Guilford  model,  which  seemed  alien  to  the  engineers. 

•  The  basic  Not-Invented-Here  syndrome  generated  suspicion  of  any  tool  coming  from  outside 
the  engineering  community. 

•  Possibly  the  strongest  negative  was  that  the  Siegel-Wolf  technique  only  simulated  the  human 
performance,  and  didn't  cover  the  performance  of  the  equipment  being  operated. 

If  he  was  to  cross  the  barrier  and  develop  a  tool  that  could  be  used  by  engineers,  Jerry  had  to 
counter  these  criticisms.  He  would  have  to  abandon  the  Siegel-Wolf  technique  and  find  a 
replacement. 

The  shopping  trip  led  him  to  examine  existing  engineering  simulations  that  he  might  use  and  adapt 
for  his  purposes.  Jerry  attended  the  1971  Winter  Simulation  conference,  going  to  tutorials  on 
various  simulation  languages.  He  considered  a  few  of  the  leading  simulations  but  found  thern 
inflexible.  They  required  special  purpose  compilers  and  would  be  expensive  and  labor-intensive  to 
adapt.  But  in  one  of  the  tutorial  sessions  he  heard  about  a  simulation  that  mimicked  the  more 
powerful  programs.  It  was  FORTRAN-based,  and  would  be  practical  to  modify  for  the  needs  of 
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his  project.  It  had  been  developed  by  Dr.  Alan  Pritsker,  an  engineer  who  had  taken  a  position  at 
Purdue  University.  Jerry  drove  to  Purdue  and  met  with  Pritsker. 

They  discussed  the  fact  that  Pritsker’s  methods  were  aimed  at  a  linear,  time-invariant  network:  the 
features  of  equipment  that  could  be  easily  represented  in  a  special  adaptation  of  the  simulation 
language  Pritsker  had  developed.  However,  Jerry  needed  to  work  with  a  nonlinear,  time-varying 
network,  namely,  the  task  operators  performed  using  the  equipment.  This  was  a  radical  idea  for  the 
time,  since  psychologists  were  using  an  information  processing  model  of  human  performance  that 
was  basically  a  linear  model.  Engineers  had  not  yet  addressed  many  problems  that  were  nonlinear 
and  time  varying.  Analysis  of  these  problems  was  difficult,  so  engineers  often  looked  at  a  series  of 
time  invariant  linear  models  to  approximate  the  more  complex  analyses. 

Pritsker  was  intrigued.  If  they  succeeded,  this  kind  of  simulation  analysis  technique  would  serve 
the  needs  of  both  the  engineering  and  psychology  communities.  Pritsker  soon  received  a  research 
contract,  with  Jerry  as  contract  monitor,  and  went  about  implementing  the  Siegel-Wolf  model 
within  his  engineering  simulation  package.  In  developing  SAINT  (Systems  Analysis  of  Integrated 
Networks  of  Tasks),  Pritsker  generalized  the  Siegel-Wolf  model  substantially. 

SAINT,  the  new  simulation,  was  used  to  represent  the  vehicle  dynamics  and  the  human  operator 
activity  for  the  B-52  refueling  task.  This  involved  two  moving  vehicles  (the  B-52  and  the  KC-135 
tanker).  The  tanker  tried  to  fly  straight  and  level.  The  B-52  pilot  pushed  the  control  yoke  and 
throttle.  The  simulation  captured  the  dynamics  of  the  system  and  the  activities  of  the  pilot  and 
aircraft.  No  one  had  ever  before  connected  a  network  simulation  with  a  continuous  process 
modelling.  The  solution  was  to  link  continuous  processes  with  states.  For  example,  altitude  is  a 
state  variable.  A  B-52  can  go  up  and  down  relative  to  the  tanker.  At  any  point  in  time  where  the 
altitude  crosses  a  line  (e.g.,  gets  too  close  to  the  tanker)  this  is  called  an  event.  The  event  will 
trigger  a  need  to  reduce  power  and  to  reduce  altitude,  which  then  changes  the  state  variable.  In  this 
way,  the  simulation  captures  the  regulation  of  a  continuous  process  (refueling)  by  a  discrete  task. 
The  outcome  is  man-machine  modelling.  The  simulation  places  a  monitor  on  the  continuous  state 
variable,  to  produce  events.  It  places  regulators  on  tasks  to  change  state  variables  (e.g.,  changing 
the  throttle  from  75%  to  50%).  This  concept  then  applies  to  other  regulators  and  state  variables  in  a 
similar  fashion. 

The  next  step  was  to  embed  into  SAINT  a  Fortran  sub-program  to  moderate  the  task  time  as  a 
function  of  other  factors,  such  as  stress,  or  anything  else.  These  were  the  psychological  levers  that 
allowed  the  simulation  to  be  used  to  calculate  how  different  conditions,  such  as  exposure  to 
radioactivity,  would  affect  performance  measures  such  as  task  time,  which  in  turn  affected 
probability  of  success.  This  last  step  was  under  the  direction  of  Deborah  Seifert  and  completed  the 
initial  version  of  SAINT,  as  summarized  in  a  1978  technical  report  by  Seifert  and  Chubb. 

To  test  the  new  simulation,  the  research  team  entered  in  a  set  of  data  on  Remotely  Piloted  Vehicles. 
SAINT  worked  very  well.  The  rigor  demanded  by  a  simulation  even  enabled  them  to  discover 
some  areas  where  the  behavioral  studies  had  not  been  done  quite  right.  In  addition,  they  were  able 
to  use  SAINT  to  show  equipment  effects  and  what-ifs  that  were  reasonable  conjectures  that  had  not 
been  previously  explored. 

SAINT  has  been  successfully  applied  to  workload  estimations  during  design  of  the  Precision 
Location  Strike  System.  One  contractor  used  SAINT  to  model  the  B-1  radar  and  establish  if  a 
multimode  radar  could  keep  up  with  the  information  processing  demands  of  the  task,  a  strictly 
engineering  analysis  application  of  SAINT.  It  has  also  been  used  to  look  at  message  traffic  and 
queuing  delays.  It  was  used  on  a  Navy  P-3  upgrade  project  to  look  at  operator  workload;  SAINT 
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verified  the  feasibility  of  the  operator  activities.  SAINT  has  been  adopted  as  a  working  tool  by 
governmental  organizations  in  other  countries  as  well  (Canada,  the  Netherlands,  Germany).  It 
helped  to  influence  the  development  of  other  simulations,  such  as  SLAM  (Simulation  Language  for 
Alternative  Modelling),  a  program  that  is  still  being  marketed.  And  it  has  given  birth  to 
MicroSAINT,  which  is  written  in  C  rather  than  Fortran.  MicroSAINT  has  been  used  in  projects 
such  as  examining  the  effects  of  performance  degradation  from  nuclear,  biological,  and  chemical 
attacks. 

SAINT  led  the  way  in  demonstrating  the  feasibility  of  a  simulation  that  can  work  through  the 
consequences  of  activity  sequences,  and  can  trace  the  way  individual  errors  can  affect  the  timing 
and  pacing  of  an  operator’s  activity,  especially  in  conjunction  with  other  operators. 
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The  Case  of  the  lUauy  lUindscreens 

An  Interview  with  H.  Lee  Task 


Experts  see  different  worlds  than  the  rest  of  us.  Where  we  may  painstakingly  try  to  figure  out  how 
to  handle  a  difficult  task,  they  can  take  one  look  and  know  how  it  ought  to  be  done.  When  we  have 
a  chance  to  watch  this  ability  in  action,  we  can  get  a  richer  sense  of  what  expertise  looks  like. 

Lee  Task  is  an  expert  in  optics.  He  received  his  master’s  degree  in  solid  state  physics  before  he 
came  to  the  Human  Engineering  Division  in  1971,  and  a  few  years  later  went  back  for  his  Ph.D.  in 
optical  science. 

The  Distorted  Windscreens 


In  1978  Major  Robert  Eggleston,  Task’s  branch  chief,  came  to  him  with  a  problem  to  solve:  the 
optical  characteristics  of  the  F-16  windscreen  were  causing  pilots  to  aim  incorrectly  and  miss  their 
targets.  When  the  manufacturers  of  aircraft  windscreens  had  shifted  from  thin  glass  windscreens  to 
thick  plastic  ones,  pilots  began  to  complain  of  distortions,  haze,  and  multiple  images  created  by  the 
new  thicker  windscreens.  These  complaints  had  been  going  on  for  years.  The  problem  of  seeing 
multiple  images  was  perhaps  the  most  frustrating  problem.  Which  target  do  I  aim  at?  Which  set  of 
landing  lights  should  I  land  on? 


The  F-16  SPO  approached  the  Human  Engineering  Division  with  this  problem  because  the 
livision  had  helped  out  oh  an  earlier  problem  with  the  F- 1 1 1  windscreen.  That  problem  had  first 
iBeen  studied  by  a  team  of  optometrists  who  had  treated  the  windscreens  as  if  they  were  like 
eyeglass  lenses.  Lee  Task;  had  shown  that  the  cmx  of  the  problem  was  in  the  refractive  and 
displacement  prOperties  of  the  windscreens  and  that  the  best  approach  was  based  on  the 
perspective  of  geometrical  optics.  Eventually,  the  manufacturers  learned  to  ittiprove  their  methods 
to  the  point  where  the  complaints  quieted  down,  even  though  the  distortions  were  still  a  nuisance. 


But  the  F-16  problem  could  not  be  ignored.  Because  no  one  had  found  a  way  to  manufacture 
distortion-free  windscreens,  the  manufacturers  had  figured  out  a  work-around.  They  were  able  to 
measure  the  unique  distortions  in  each  windscreen,  and  use  these  measurements  to  feed  corrections 
into  the  Heads-Lfp  Display  (HUD)  to  compensate  for  the  discrepancies. 

Now  the  problem  had  shifted.  While  this  measurement  strategy  worked,  it  seemed  very  labor 
intensive.  The  method  used  at  the  time  for  evaluating  the  optical  aiming  error  of  the  windshield 
required  two  man-days  of  effort.  A  laser  was  directed  through  the  windscreen  onto  a  wall  100  feet 
away  where  a  technician  would  record  the  movement  of  the  laser  shot.  This  method  was  replicated 
for  60  data  points.  Due  to  its  manual  nature  the  method  was  prone  to  error  at  many  stages  of  the 
data  collection  process. 

When  Task  heard  about  how  these  measurements  were  being  made  he  could  not  believe  the 
inefficiency  of  the  method.  “Why  are  they  doing  that?”  he  wondered.  He  immediately  knew  there 
was  a  more  efficient  way  of  collecting  the  windscreen  data  using  more  up  to  date  techniques  and 
equipment. 
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Task's  idea  incorporated  placing  the  windscreen  on  a  movement  table  which  would  rotate  the 
windscreen  through  known,  accurate  increments.  A  laser  (later  replaced  by  an  incandescent  light 
source)  was  projected  through  the  windscreen  to  a  recording  device  just  two  feet  from  the 
transparency.  This  device  allowed  over  a  thousand  data  points  to  be  collected  in  an  hour,  providing 
the  curve  fit  data  from  four  different  "eye-points:"  the  HUD  position,  the  left  and  right  eye- 
positions  of  the  pilot,  and  the  gun  camera  viewpoint. 

Task  built  the  first  prototype  of  the 
measurement  device  in  the  basement  of  one  of 
the  Armstrong  Laboratory  buildings  at 
WPAFB  and  the  SPO  had  the  windscreen 
manufacturers  ship  all  their  windscreens  to  the 
basement  for  measurement  until  they  could  get 
hold  of  their  own  measurement  devices. 

Task  patented  the  system  with  several 
co-workers.  In  1979,  they  formed  a  company 
and  began  to  sell  the  measurement  devices  to 
the  windshield  manufacturers  who  were  unable  to  handle  the  electronics  requireinents  to  make  their 
own  devices.  The  endeavor  led  to  sales  all  over  the  world.  The  endeavor  was  a  richer  learning 
experience  and  success  than  a  financial  one  as  the  market  was  soon  saturated.  There  s  not  too 
much  call  for  aircraft  windscreen  measurement  devices.  But  the  benefits  to  the  Air  Force  pilots 
have  been  immeasurable. 


The  estimated  improvements  from  this  system 
include  drastic  reductions  in  man-hours  spent 
taking  me^uretnentt;.  There  was  also  an 
eStMated  1 5-20%  improvement  in  data  ^curacy 
from using  1000  lata  points  versus  60  making 
up  the  curve,  and  a  more  accurate  automated 
windshield  rotatibn  device,  as  well  as  four  eye- 
point  measures  instead  of  just  one. 
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The  Pilots  luho  had  to  be  Kept  in  the  Dark 

An  Interview  with  H.  Lee  Task 


Landing  an  aircraft  in  the  dark  has  never  been  easy.  Landing  an  aircraft  in  the  dark  on  a  makeshift 
runway  using  night  vision  goggles  (NVGs)  was  a  real  headache.  This  was  a  problem  handed  to 
Task  and  his  colleagues  by  Military  Airlift  Command  (MAC)  in  early  1982. 

The  problem  was  that  the  transport  pilots  operating  on  austere  runways  were  having  problems 
judging  the  glide  slope  while  landing  on  these  temporary,  covert  landing  strips.  (These  were  often 
landing  strips  in  hostile  territory.)  The  transport  pilots  would  come  in  for  landing  wearing  NVGs 
which  reduced  their  ability  to  judge  their  angle  of  descent  relative  to  the  landing  strip.  This  had 
already  resulted  in  at  least  one  C-130  crash. 

Task  was  acknowledged  as  one  of  the  experts  at  WPAFB  in  the  use  of  NVGs.  This  was  simply 
because  he  was  one  of  the  first  people  at  WPAFB  to  have  a  pair.  He  had  been  given  a  set  of  PVS-5 
NVGs  in  the  late  70s  by  a  friend,  an  Army  officer  who  had  been  a  former  colleague  at  the  Human 
Engineering  Division.  Task  had  optically  characterized  the  equipment  following  which  the  goggles 
had  sat  on  a  shelf  gathering  dust. 


MAC  had  learned  that  Task  and  a  colleague,  Lee  Griffin,  were  knowledgeable  about  NVGs,  and 
had  sought  their  help  on  earlier  occasions.  Once,  MAC  had  needed  compatible 
electroluminescent  lighting  for  covert  austere  runway  landing  strip  lights,  Another  time  they 
were  having  problerns  with  NVGs  and  the  in-cockpit  lighting.  Task  and  Griffin  had  solved  their 
problems  and  so  now  Task  was  the  person  they  turned  to  after  the  C-130  crash. 


MAC  wanted  Task  to  come  up  with  a  device  which  could  indicate  whether  the  pilot  was  on  the 
glide-slope,  too  high  or  too  low,  while  he  was  wearing  NVGs.  All  this  had  to  be  done  in  a  way 
that  didn't  compromise  the  position  of  the  special  operations  ground  force. 

Task  realized  that  the  problem  was  solvable.  He  merely  had  to  specify  some  of  the  parameters  in 
order  to  work  out  the  solution.  He  identified  the  problem  as  having  three  key  features: 

•  He  would  need  a  light  that  was  bright  enough  to  be  seen  through  NVGs  at  long  distances 
without  giving  away  the  ground  position. 

•  He  would  also  need  to  be  able  to  indicate  the  optimum  glide-slope  and  whether  the  pilot  was 
deviating  above  or  below  that  glide-slope  angle. 

•  Finally,  the  whole  system  would  need  to  be  easily  and  cheaply  made,  small  enough  to  be 
carried  by  the  ground  troops,  and  quickly  and  accurately  assembled. 

The  first  problem  he  addressed  using  the  old  NVGs  which  were  still  sitting  on  his  shelf.  One  cold 
night  he  and  his  colleagues  fixed  up  some  old  flashing  car  tail  lights  at  one  end  of  an  inactive 
taxi  way  on  Area  B.  They  drove  about  a  mile  to  the  far  end  of  the  taxiway  to  see  if  they  could  see 
the  lights  through  the  NVGs.  (While  they  were  driving  Task  decided  to  switch  off  his  own  car 
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lights  and  drive  using  the  NVGs  just  to  see  what  it  was  like.  Within  seconds  they  were  pulled  over 
by  the  base  security  police!  Task  explained  that  they  were  testing  the  NVGs  and  the  policeman 
backed  off,  satisfied  with  Task's  response.) 

The  flashing  lights  at  the  far  end  of  the  base  were  clearly  visible.  The  best  blink  rate  seemed  to  be 
at  a  rate  of  about  one  flash  every  two  seconds  (0.5  Hz).  To  keep  the  lights  from  being  seen  by 
unwanted  eyes  they  had  housed  them  in  a  box  with  a  single  aperture  and  an  infrared  filter  so  that 
only  an  approaching  pilot  could  see  them. 

The  next  problem  was  to  establish  how  the  lights  would  indicate  a  glide  slope.  One  set  of  lights 
would  be  placed  on  each  side  of  the  runway.  Task  set  them  up  so  that  the  system  could  be  set  to 
several  different  glide  slopes  for  different  situations.  At  the  correct  glide  slope  the  pilot  would  see 
one  flashing  light  and  one  static  light  (one  on  each  side  of  the  runway).  If  the  pilot  deviated  above 
the  glide  slope  he'd  see  two  flashing  lights,  and  if  he  deviated  below  the  glide  slope  he'd  see  two 
steady  lights. 

The  final  problem  was  overcome  by  producing  a  unit  that  fit  into  a  briefcase-sized  c^ing  case. 
The  materials  used  for  the  system  could  all  be  easily  obtained.  And  the  design  made  it  easy  to  set 
up  the  system.  The  lights  could  be  aligned  flat  to  the  ground  using  a  bubble  level,  and  different 
glide-slopes  could  be  set  using  wedges  cut  to  the  correct  angle  and  aligned  on  the  light  units. 

The  bread  board  units  were  initially  flight  tested  in  April  of  1982  to  see  if  the  pilots  could  acquire 
the  indicators  through  NVGs  at  sufficient  distances,  to  see  if  the  power  was  sufficient,  and  to 
assess  whether  the  system  could  be  spotted  by  ground  troops  who  might  comprorpise  the  position. 

In  July  1982  the  prototype  units  were  officially  and  successfully  flight  tested  at  Pope  AFB. 
(During  the  flight  test.  Task  noticed  something  else  that  was  unusual  about  the  way  the  pilots  were 
landing,  but  that’s  another  story.  See  The  Pilots  who  had  to  Play  it  by  Ear — An  Inte^iew  with  H. 
Lee  Task.)  The  special  operations  liaisons  asked  for  twelve  units  for  operational  use  immediately. 
So  the  project  transferred  from  a  problem  statement  to  an  application  in  operational  use  within  six 
months. 


The  acquisition  of  twelve  units  for  irninedmte  use  by  special  forces  was  accomplished  through 
unconventional  channels  due  the  nature  of  the  "black"  project  for  covert  operatiops.  The  twelve 
Units  were  produced  under  the  official  title  of  Operational  Test  and  Evaluation  units.  Had  the 
official  procurement  route  been  taken  it  may  have  taken  several  years  to  get  this  relatively  small 
project  in  place.  Not  everyone  was  happy  with  this  rapid  turnaround.  MAC  was  written  up  by  the 
Inspector  General  for  not  following  the  correct  procurement  channels! 


Although  no  formal  evaluation  of  the  project  has  been  conducted,  it  can  be  assumed  that  these 
simple  devices  have  improved  the  safety  of  the  transport  pilot  landings  and  guaranteed  the  safe 
delivery  of  important  supplies  to  the  ground  forces  relying  on  them. 
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The  Pilots  who  had  to  Play  it  hy  Ear 

An  Interview  with  H.  Lee  Task 


In  July  1982,  while  videotaping  the  final  flight  tests  of  the  glide-slope  indicator  (see  The  Pilots 
who  had  to  be  Kept  in  the  Dark  -  An  Interview  with  H.  Lee  Task)  from  the  flight  engineers  seat  of 
the  C-130,  Task  made  an  interesting  discovery  which  led  to  another  important  project  for  his 
group.  Task  had  been  lining  up  his  bulky  videocamera  with  a  pair  of  NVGs  and  the  glide-slope 
indicator  on  the  ground  in  order  to  video  what  the  pilot  was  seeing.  Task  also  had  the  camera 
hooked  up  to  pick  up  the  pilot's  comments  and  intra-cockpit  communications  for  the  video. 

As  the  pilot  came  in  to  land,  Task  noticed  that  the  navigator  and  co-pilot  were  doing  a  lot  of  talking 
to  the  pilot,  providing  information  about  range  to  the  landing  strip,  altitude,  sink  rate,  and  air 
speed.  Task  suddenly  realized  that  the  pilot  was  completely  isolated  from  any  displays  in  the 
cockpit  while  he  was  "heads-up"  looking  towards  the  landing  strip.  The  pilot  was  relying  on 
critical  information  from  his  teammates  in  order  to  avoid  crashing  into  the  ground  or  running  out  of 
air  speed.  He  had  to  integrate  the  running  stream  of  numbers  to  maintain  some  sort  of  situation 
awareness. 


When  Task  returned  to  Wright-Patt,  he  pointed  out  this  observation  to  Charlie  Bates,  the  head  of 
the  division,  and  to  Tom  Furness  (a  branch  chief  in  the  division)  during  the  flight  test  debrief.  As 
soon  as  he  realized  that  the  crew  was  reading  the  pilot's  instruments  out  loud,  TaSk  recognized 
that  this  critical  information  Could  be  provided  directly  into  the  NVG  device.  The  big  question  was 
whether  the  synaboiogy  would  distract  the  pilot  from  the  outside  view,  or  even  obsfcure  the  outside 
wewi 


Bates  and  Furness  left  Task  to  come  up  with  a  workable  solution.  Task  drew  up  a  sketch  of  his 
envisioned  solution  in  August  1982.  He  then  went  down  to  his  machinist  and  told  him  what  he 
wanted  in  order  to  provide  a  proof  of  concept.  The  machinist  put  together  a  housing  for  an  optical 
system  that  attached  to  the  front  of  the  NVG  lens.  A  transparent  representation  of  a  HUD 
symbology  photograph  was  attached  to  the  contraption  so  that  the  symbology  was  reflected  in  the 
lens  in  front  of  the  NVGs  but  which  also  allowed  light  from  the  outside  world  to  produce  the  night 
vision  image.  The  result  was  that  the  HUD  symbology  was  superimposed  on  the  NVG  image.  In 
theory  this  provided  flight  information  directly  to  the  pilot. 

In  January  1983,  Task  showed  this  to  Bates  and  Furness.  This  proof  of  concept  was  enough  to 
persuade  them  to  fund  Task  to  set  up  a  dynamic  mock-up  run  from  an  old  Apple  computer.  This 
demonstration  showed  that  the  imagery  from  the  HUD  and  the  filtering  of  light  by  the  semi- 
reflective  beam  splitter  in  front  of  the  NVGs  would  not  interfere  with  clarity  of  the  NVG  image. 
The  pilots  could  see  both  the  outside  scene  and  the  displayed  symbology. 

By  April  1983  Task  had  the  mock  up  slaved  to  the  pilot's  head  position  so  that  the  imagery 
represented  the  pilot's  frame  of  reference  rather  than  being  referenced  to  the  aircraft's  heading.  The 
system  was  demonstrated  to  MAC,  which  liked  the  idea  of  the  symbology  in  the  NVG,  but  wasn't 
sold  on  the  head-slaved  aspect. 
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In  the  fall  of  1983  the  NVG  HUD  was  successfully  flight  tested  on  a  C-141.  The  system  has  been 
fitted  to  many  aircraft  in  the  U.S.  arsenal  and  is  now  a  standard  piece  of  equipment.  The  pilot  is 
now  able  to  see  critical  flight  information  while  wearing  the  vision-restricting  NVGs,  and  does  not 
need  to  rely  on  that  information  being  relayed  verbally  from  other  members  of  the  flight  crew. 

Lee  Task  had  obtained  numerous  patents  while  working  fbr  the  Human  Engineering  Division,  but 
this  was  one  that  got  away.  He  had  applied  for  the  patent,  but  the  administrator  in  charge  of 
receiving  the  application  rejected  it,  claiming  that  it  was  not  sufficiently  different  fiiOm  bthef 
devices.  Task  knew  that  this  was  incorrect,  but  he  was  too  busy  with  other  projects  to  file  a 
rebuttal  and  the  opportunity  slipped  away. _  ' ;  ■  .V: 
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UJarrick's  Principle:  The  Case  of  the 
Compatible  Controls 


An  Interview  with  Mel  Warrick 


The  room  temperature  is  a  bit  too  high.  The  thermostat  shows  that  it's  set  for  78°.  Which  way 
should  I  turn  the  knob  to  make  it  cooler?  My  vertical  velocity  indicator  is  showing  that  my  vertical 
velocity  is  a  little  slow.  Which  way  should  I  move  the  control  wheel  to  increase  it? 

Mel  Warrick  wanted  to  know  how  people  move  controls  in  such  situations.  So  he  set  about 
studying  what  people's  preferences  for  control  movement  are  and  ended  up  with  a  law  named  after 
him.  Some  ideas  are  so  simple  you  wonder  if  you  were  the  only  one  to  think  of  it.  And  then  to  get 
a  principle  about  it  named  after  you,  well  that's  laughable.  That's  Mel's  take  on  the  whole 
Warrick's  Principle  idea. 

An  Ingenious  Contraption 

It  seemed  pretty  obvious  to  Mel  what  should  happen  if  you  want  a  display  indicator  to  move  one 
way  and  you  have  a  dial  to  control  the  direction  of  motion  of  the  indicator,  but  common  sense  isn't 
always  so  "common." 

In  1946,  Mel  had  been  invited  by  Paul  Fitts  to  join  him  at  the  Air  Force's  Aero  Medical  Laboratory 
at  Wright-Patterson.  Mel's  background  as  a  science  teacher,  amateur  electronics  buff,  air  force 
bombardier,  and  experience  in  personnel  and  training  issues  intrigued  Fitts,  who  thought  Mel 
would  fit  well  into  the  engineering  psychology  research  program. 

Following  a  study  looking  at  the  readability  of  dials  under  "excess"  accelerations  (2-3  G)  using  the 
hand  propelled  centrifuge,  Mel  began  a  study  to  address  the  stereotypes  for  control-display 
movement  relationships.  Fitts  and  Warrick  had  been  wondering  about  the  ways  that  controls  work 
and  the  expectancies  of  the  operators  of  these  controls. 

Their  awareness  of  these  issues  had  been  raised  by  attempts  to  understand  the  differences  between 
British  and  American  stereotypes  of  control  operation  in  aviation  and  more  generally  in  everyday 
devices  such  as  light  on/off  switches.  They  had  also  been  asked  by  an  engineering  lab  on  the  base 
to  look  at  the  adjustment  controls  in  aircraft  which  moved  the  pilot  closer  to  the  rudder  controls. 
The  engineering  lab  wanted  to  know  which  way  to  move  the  control  for  adjusting  the  pilot's 
position  closer  to  or  further  away  from  the  rudder  pedals. 

Mel's  interest  in  the  control/display  compatibility  area  was  as  much  driven  by  an  interest  in  the 
building  of  the  experimental  apparatus  as  it  was  in  the  study  of  the  phenomena  themselves.  This 
personal  interest  in  electronics  and  gadgets  had  started  as  a  young  boy  and  had  included  the 
building  of  a  set  of  wings  for  his  bicycle. 

Mel  set  about  building  what  he  calls  his  "ingenious"  contraption.  Mel  wanted  to  see  his  subjects' 

During  a  test  ''flight"  he  had  actually  succeeded  in  getting  the  vehicle  off  the  ground.  The  key 
lesson  he  had  learned  from  that  experience  was  the  need  for  something  to  maintain  stability  and 
control  of  his  vehicle  once  he  was  airborne.  The  learning  of  this  lesson  was  almost  as  hard  as  the 
ground  on  which  he  "landed!" 
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preferences  for  moving  a  control  knob  to  elicit  a  control  response.  He  built  an  apparatus  which  had 
a  control  knob  located  underneath  the  middle  light  of  a  row  of  five.  When  one  of  the  lights  was  on, 
the  subject  had  to  move  the  knob  in  order  to  bring  the  light  back  to  the  middle  of  the  five  lights. 
What  the  subjects  did  not  know  was  that  whichever  way  they  turned  the  knob,  the  light  would 
always  come  back  to  the  center. 

The  results  of  the  study  indicated  that  subjects  turned  the  knob  so  that  the  knob  was  moving  in  the 
same  direction  as  the  expected  light  movement.  So,  if  the  light  on  the  far  right  was  on,  the  subject 
would  turn  the  knob  counter-clockwise  (to  the  left)  to  bring  the  light  back  to  the  center. 

Not  so  "Common"  Sense 


Warrick  thought  he’d  merely  pointed  out  the  obvious.  However,  things  were  not  as  simple  as  they 
first  appeared.  He  had  discovered  that  if  you  want  to  move  the  lights  to  the  left  you  move  the  dial 
to  the  left.  But  what  if  the  control  knob  is  placed  above  the  set  of  lights,  does  the  same  simple  rule 
apply?  No,  it  did  not.  In  this  case,  if  they  wanted  to  move  the  light  to  the  left,  subjects  turned  the 
dial  to  the  right.  What  was  happening  here?  The  position  of  the  control  relative  to  the  display  had 
changed,  and  so  had  the  movement  relationship.  In  other  words,  the  bottom  of  the  dial  now  moved 
to  the  left.  Mel  adapted  the  device  to  a  number  of  configurations  to  further  test  the  problem.  In  one 
case  the  lights  were  arranged  in  an  arc  around  the  control  dial,  in  another  case  the  knob  was  located 
on  the  side  of  the  box  with  the  lights  on  the  facing  side. 


This  phenomenon  is  explained  by  what  is  now  known  as  the  Warrick  principle  which  asserts  that 
the  closest  part  of  the  moving  element  of  the  control  should  move  in  the  same  direction  as  the 
closest  part  of  the  moving  element  of  the  display,  as  if  there  was  a  mechanical  linkage  between 
the  two. 


Warrick's  demonstration  was  followed  up  with  paper  and  pencil  tests  and  what  he  calls  operational 
interviews.  For  these  sessions,  systems  were  set  up  in  operational  settings  and  actual  operators 
would  use  them  and  report  their  preferences. 


Warrick  never  actually  named  His  discoveiy  .  It  wasn't  until  later  that  this  display/control 
relationship  principle  was  formerly  named  after  him,  in  fact  he  didn't  even  know  it  had  been 
named  as  such  until  fairly  recently!!  His  discovery,  along  with  others  from  the  lab,  pointed  out 
the  need  for  designers  to  observe  not  only  how  they  designed  their  displays  and  controls  but  also 
where  they  put  them,  and  how  they  made  them  move  relative  to  one  another. 


For  Further  Reading 

Warrick,  M.  (1947).  Direction  of  movement  in  the  use  of  control  knobs  to  position  visual 
indicators  (USAF  AM C  Report  No.  694-4C).  Wright  AFB:  U.S.  Air  Force.  (Also  published  in  P. 
M.  Fitts  (Ed.),  Psychological  research  on  equipment  design.  Washington,  DC:  U.S.  Government 
Printing  Office,  Army  Air  Forces  Aviation  Psychology  Program  Research  Reports,  Report  19.  Pp. 
137-146.) 
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DISCUSSION 


In  reviewing  the  case  studies,  a  number  of  themes  emerge.  One  is  the  contrast  between  projects 
undertaken  with  specific  users  in  mind,  versus  projects  intended  to  have  a  broader  application.  A 
second  theme  concerns  the  way  the  research  projects  were  identified  and  pursued  from  the  outset. 
A  third  theme  is  the  type  of  organizational  support  needed  by  these  projects. 

User-centered  versus  Generic  Projects 

One  way  to  categorize  the  projects  described  in  this  report  is  whether  they  were  attempting  to 
support  a  specific  user  community  or  not  (this  distinction  is  similar  to  the  distinction  recognized  in 
the  Armstrong  Laboratory  as  fire  fighting  versus  fire  prevention).  The  projects  that  were  directed  at 
a  specific  user  community  focused  on  a  single,  clearly  defined  user,  and  a  single,  clearly  defined 
need.  Earl  Sharp’s  work  for  the  B-52  community  and  Phil  Kulwicki’s  activities  for  the  F-15  and 
F-16  SPO  fit  into  this  category.  So  do  Lee  Task’s  efforts  for  the  F-16  SPO  regarding  windscreens, 
and  for  the  Military  Airlift  Command  regarding  night  vision  landings.  The  work  of  Topmiller  and 
Heckart  also  seems  to  belong  in  this  category,  since  the  human  factors  standards  they  developed 
were  intended  for  the  Ballistic  Missile  Division,  even  though  the  scientists  hoped  that  the  human 
factors  standards  would  be  used  by  other  Air  Force  weapons  development  programs  as  well. 

For  the  projects  that  had  a  clear  user  in  mind,  sometimes  the  need  was  narrow  and  specific.  One 
example  is  in  the  case  of  the  fat  cursors  where  Earl  Sharp's  research  team  helped  work  out  a  better 
type  of  cursor  design.  Other  times  the  research  program  was  more  broadly  conceived  to  help  a  user 
community  handle  a  new  type  of  challenge. 

Other  projects  were  not  aimed  at  any  specific  user  community.  Dave  Post’s  development  of 
miniature  color  displays  does  not  solve  any  immediate  operational  problem.  Charlie  Clauser’s 
effort  to  define  the  moment  of  inertia  of  body  segments  had  a  vague  user  (someone  interested  in 
working  on  orientation  in  zero  gravity  environments)  but  certainly  no  one  was  waiting  with  a 
pressing  need  for  this  research  to  be  completed.  Further,  the  eventual  uses  of  these  measurements 
had  little  to  do  with  locomotion  in  space.  Bob  O’Donnell’s  reaction  to  the  request  to  “measure  me 
workload”  also  fits  this  category.  The  initial  concept  of  helping  designers  anticipate  workload  has 
still  not  really  materialized;  one  of  the  primary  uses  thus  far  is  to  measure  the  effect  of  chemical  and 
other  stressors  on  cognitive  processes. 

The  time  course  of  impact  is  different  for  the  two  categories  of  projects.  The  projects  directed  at 
specific  users  have  an  impact  that  is  surprisingly  fast.  One  of  Earl  Sharp’s  projects,  the  case  of  the 
fat  cursors,  took  only  four  months  from  detection  of  the  need,  through  redesign,  through  empirical 
testing,  to  recommendations.  We  suspect  that  there  were  many  other  'projects'  that  were  even 
faster,  perhaps  just  a  telephone  conversation  in  which  advice  and  guidance  were  offered  to  a 
customer  who  needed  a  simple  fix.  In  contrast,  the  generic  projects  that  were  not  intended  to 
support  specific  users  were  much  slower,  typically  taking  several  years  and  showing  a  slower  pace 
during  the  initial  phases. 

The  degree  of  impact  is  difficult  to  compare.  Some  of  the  projects  to  help  out  specific  users  did  not 
have  as  great  an  impact  outside  of  these  user  communities,  although  it  is  difficult  to  trace  the  nature 
of  influences.  Sharp’s  work  for  SAC  inspired  others  to  set  up  comparable  simulation  facilities  for 
other  types  of  aircraft.  Other  user-centered  projects  have  uncovered  general  principles  that  had  a 
wider  impact.  Kulwicki’s  studies  of  high-G  performance  are  of  general  interest.  Task’s  work  on 
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windscreen  evaluation,  and  on  getting  symbology  into  Night  Vision  Goggles,  also  has  enjoyed 
widespread  application.  The  work  of  Topmiller  and  Heckart  easily  transitioned  into  standards  that 
could  be  used  outside  the  Ballistic  Missile  Command.  We  can  see  from  the  cases  presented  here 
that  user-centered  research  sometimes  has  a  wider  application  beyond  the  user  group  originally 
targeted. 

The  generic  projects,  without  specific  users  in  mind,  were  intended  to  have  a  wider  impact  from 
the  very  beginning.  That  was  the  rationale  for  conducting  these  projects  in  the  absence  of  an 
identified  sponsor.  Post’s  work  on  miniaturized  color  displays  can  be  used  in  many  different 
ways.  In  the  final  analysis,  it  is  not  clear  if  the  Air  Force  will  gain  more  from  these  miniaturized 
color  displays  or  from  Earl  Sharp’s  solution  to  the  B-52  cursor  problem.  The  Human  Engineering 
Division  has  pursued  both  user-centered  and  generic  projects  in  an  attempt  to  support  immediate 
operational  needs  and  to  anticipate  the  needs  of  the  future. 

Trying  to  Understand  the  Rainbow 

It  is  only  within  the  last  decade  that  the  physics  of  rainbows  has  been  satisfactorily  understood.  Yet 
scientists  have  been  trying  to  understand  rainbows  for  at  least  300  years.  Some  questions  cannot 
be  answered  at  a  given  level  of  technology.  Even  though  the  question  seems  important,  little  can  be 
accomplished  and  the  scientific  inquiry  is  largely  a  waste  of  time.  Only  when  the  right  analytical 
techniques  have  been  developed  can  we  expect  to  make  progress.  It  is  not  a  mistake  to  tty  to 
understand  the  rainbow,  but  it  can  be  a  mistake  to  persist  before  the  necessary  tools  are  in  place. 

The  researchers  described  in  these  case  studies  seem  to  have  a  knack  for  picking  problems  that  are 
important  and  have  just  become  solvable.  They  were  not  wasting  their  time.  They  were  able  to 
harness  new  developments  and  take  advantage  of  opportunities.  When  Lee  Task  heard  about  the 
way  manufacturers  were  taking  measurements  on  F-16  windscreens,  his  immediate  reaction  was 
that  there  was  a  much  more  efficient  method,  using  lenses.  Lee  had  just  completed  his  Ph.D.  in 
optical  science.  Five  years  earlier,  before  his  Ph.D.  experience,  he  might  not  have  seen  the 
opportunity,  and  he  might  not  have  chosen  to  work  on  the  project.  Phil  Kulwicki  became  interested 
in  high-acceleration  cockpits  in  part  because  he  knew  about  ways  of  conducting  simulations  that 
would  not  take  tremendous  effort.  Charlie  Clauser’s  interest  in  measuring  moments  of  inertia  was 
coupled  to  his  knowledge  that  the  relevant  techniques  had  already  been  demonstrated  with 
baboons.  In  Bob  O’Donnell’s  case,  the  cognitive  measures  approach  only  made  sense  because  of 
recent  developments  in  the  field,  suggesting  a  set  of  tools  that  could  be  readily  adapted  to  measure 
cognitive  aspects  of  workload.  Ten  years  earlier  the  project  would  not  have  made  much  sense. 

Dave  Post’s  commitment  to  miniaturizing  color  displays  was  based  on  his  discovery  that  there  was 
a  plausible  approach  using  color  subtraction.  Otherwise  he  would  have  turned  to  other  projects 
when  he  ran  into  funding  problems. 

All  the  cases  that  we  chose  to  focus  on  illustrate  the  ability  to  recognize  the  solvability  of  an 
important  problem.  Of  course,  these  are  a  sample  of  a  large  number  of  stories,  not  all  of  which 
may  have  had  such  great  operational  impacts.  We  have  no  baseline  against  which  to  compare  hit 
rates.  It  is  hypothesized  that  the  skill  of  these  individuals  is  in  the  seizing  of  opportunities  to  pursue 
higher  probability  pay-offs. 

Many  of  these  case  studies  show  the  importance  of  problem  finding.  Lee  Task  hears  the  co-pilot 
and  navigator  reading  out  instrument  data  to  the  pilot  and  finds  a  problem  he  can  solve.  Phil 
Kulwicki  learns  about  high  acceleration  cockpits  and  finds  a  set  of  problems  he  can  solve.  Don 
Topmiller  and  Steve  Heckart  try  to  use  human  factors  guidelines  in  evaluating  new  ballistic  missile 
systems  and  find  a  problem.  No  one  brought  the  problem  to  these  researchers.  They  discovered  the 
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problems  and  identified  them  as  worth  pursuing  because  they  could  anticipate  that  a  solution  was 
feasible.  Each  case  seems  accidental.  Earl  Sharp  visits  a  friend  and  sees  videotapes  of  B-52 
Electronic  Warfare  Officers  at  work,  wrestling  with  a  poorly  laid  out  work  station.  Lee  Task  goes 
along  for  a  night  landing  to  test  his  glideslope  indicators  and  listens  to  the  pilot  getting  talked 
through  the  landing.  Don  Topmiller  is  horrified  to  see  mechanics  handling  volatile  chemicals 
without  wearing  gloves.  To  put  these  cases  into  context,  we  should  remember  that  the  researchers 
were  also  exposed  to  many  other  situations,  and  observed  many  other  types  of  difficulties,  and  did 
not  elect  to  pursue  those.  If  we  envision  the  problem  identification  process  as  a  key,  then  the  two 
tumblers  are:  seeing  a  problem  worth  solving,  and  seeing  an  opportunity  to  solve  it.  When  these 
two  criteria  are  met,  and  the  two  tumblers  are  lifted,  the  research  engine  can  start. 

In  other  cases,  the  problem  was  already  identified.  Users  complained  about  inefficient  ways  of 
measuring  F-16  windscreens,  or  cursors  that  obscured  the  targets.  So  one  tumbler  had  already 
been  lifted.  The  researchers  needed  to  feel  confident  that  the  other  tumbler  could  be  managed — ^that 
there  was  a  plausible  strategy,  an  opportunity  to  be  exploited — ^before  becoming  engaged  with  the 
project. 

Anthropologists  distinguish  between  farming  cultures  and  hunter-gatherer  cultures.  These  cases 
show  researchers  who  are  hunters  and  gatherers.  They  are  not  setting  out  to  build  multi-year 
programs.  They  are  trying  to  solve  problems  quickly,  using  whatever  resources  they  can  quickly 
find.  Some  of  them,  like  Earl  Sharp  and  Lee  Task,  use  their  accent  into  the  machine  shops  on  base 
to  quickly  fabricate  pieces  of  equipment.  Phil  Kulwicki  starts  out  by  scavenging  software 
programs  he  can  use  to  run  simulations.  Earl  Sharp  scavenges  a  B-52  simulator.  The  equipment  is 
a  means  to  a  solution,  not  an  end  in  itself.  The  case  studies  show  a  frequent  theme  of  finding  and 
adapting  methodologies,  technologies,  and  data  to  serve  the  problems  at  hand. 

A  commonality  across  the  case  studies  is  that  the  researchers  were  trying  to  have  an  impact,  rather 
than  waiting  to  receive  instructions  and  follow  someone  else’s  research  agenda.  Even  when 
someone  brought  a  problem  to  the  attention  of  the  researchers,  they  took  on  the  initiative  to  find  a 
solution.  This  initiative  was  particularly  striking  when  a  project  ran  into  funding  difficulties,  and 
the  researcher  had  to  scavenge  for  funds.  The  initiative  extended  to  cases  where  rules  had  to  be 
bent.  As  Earl  Sharp  stated,  it  is  easier  to  ask  forgiveness  than  to  ask  permission. 

Collaborating  Communities 

No  one  can  succeed  in  an  environment  such  as  the  Human  Engineering  Division  without  being  part 
of  teams  and  networks.  The  nature  of  the  team  varied  in  the  different  cases.  Sometimes,  the  team 
was  made  up  of  other  researchers  at  the  Division.  Topmiller  and  Heckart  were  part  of  a  team  that 
included  Charlie  Bates,  Dave  Greek,  Lt  Kibler,  Ken  Kennedy,  and  Milt  Alexander.  The  members 
of  this  team  recall  that  period  of  research  with  great  affection,  as  a  time  of  shared  enthusiasm  and 
purpose.  O’Donnell’s  team  was  primarily  the  three  groups  of  researchers  working  underneath  him. 
Charlie  Clauser’s  team  identity  included  people  in  the  Division,  but  was  centered  around 
professional  colleagues  and  a  network  that  began  in  graduate  school.  His  effort  to  measure 
moments  of  inertia  was  a  joint  project  with  colleagues  at  the  Civil  Aero  Medical  Institute  in 
Oklahoma  City.  Dave  Post’s  project  also  depended  on  his  network  of  professional  colleagues. 

In  contrast,  Earl  Sharp’s  network  centered  around  the  B-52  SPO  and  the  user  groups.  Earl 
depended  on  the  people  in  his  branch,  but  his  primary  allegiance  appeared  to  be  to  the  users  rather 
than  to  the  Division.  Phil  Kulwicki ’s  network  was  centered  around  the  advanced  fighter 
community,  primarily  the  SPOs,  and  the  avionics  group. 
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It  is  tempting  to  speculate  that  user-centered  research  requires  tight  links  outside  the  laboratory,  and 
generic  projects  requires  tight  links  inside  the  laboratory.  But  there  are  counter-examples.  Lee 
Task’s  projects  are  as  user-centered  as  any  in  this  report,  yet  he  has  not  forged  the  same 
relationship  to  any  user  community  as  have  Sharp  and  Kulwicki.  Lee  Task’s  team  was  a  small 
group  at  Armstrong  Laboratory  that  helped  him  put  together  the  equipment  he  needed  and  cany  out 
the  demonstrations.  Another  hypothesis  is  that  engineers  are  more  likely  to  forge  alliances  with 
user  communities,  whereas  physicists  and  behavioral  scientists  are  more  likely  to  affiliate  with  their 
professional  communities.  There  does  not  seem  to  be  any  simple  generalization  about  who  will 
form  ties  with  a  user  community,  and  who  will  maintain  primary  ties  to  the  laboratory  team. 
Credibility  with  users  seems  to  emerge  independent  of  domain  or  educational  speciality. 

The  Role  of  Management 

The  role  of  upper  management  is  conspicuous  in  these  case  studies  by  its  absence.  The  term 
“benign  neglect”  comes  to  mind.  The  researchers  find  their  own  problems,  form  connections  with 
users,  get  outside  funding  for  small  demonstration  efforts,  then  get  larger  commitments  from  the 
sponsors.  One  common  refrain  was  appreciation  that  upper  management  didn't  interfere.  This 
appreciation  was  sincere.  The  researchers  were  not  in  an  adversarial  role  with  management.  They 
seemed  happy  with  the  people  who  ran  the  Division,  and  grateful  for  support.  But  they  were  also 
grateful  that  management  did  not  require  them  to  spend  excessive  time  and  energy  justifying  their 
programs  and  documenting  their  progress.  Those  who  later  moved  into  managerial  positions 
expressed  a  reduction  in  job  satisfaction  and  enthusiasm  as  they  lost  contact  with  technical 
programs.  In  general,  the  researchers  were  pleased  to  be  treated  as  professionals,  ^nd  to  be  given 
the  latitude  to  follow  their  judgment  about  which  problems  to  address.  They  felt  that  the  non- 
intrusive  atmosphere  in  the  Division  was  important  for  their  discovery  processes.  Further,  even 
when  the  researchers  were  bending  rules  they  knew  that  management  trusted  their  judgment. 

Only  in  two  of  these  cases  did  the  initial  problem  finding  come  from  upper  management  at  the 
Human  Engineering  Division,  rather  than  from  the  researchers  themselves  or  from  the  user 
community.  One  was  the  request  that  Dave  Post  look  into  miniaturizing  color  displays,  and  the 
other  was  the  request  that  Bob  O’Donnell  find  a  way  to  measure  workload.  The  first  was 
successful  because  Post  took  on  initiative  after  seeing  that  there  was  an  opportunity,  using  color 
subtraction.  The  second  was  successful  in  a  more  roundabout  way — the  cognitive  test  set  is  not 
being  used  to  measure  workload  in  order  to  help  equipment  designers,  but  rather  to  measure  the 
effects  of  stress  and  chemical  substances  on  cognitive  performance.  Other  than  these  two 
exceptions,  the  problem  finding  was  not  directed  by  upper  management. 

Another  function  of  management  was  to  create  a  cooperative  atmosphere.  We  commonly  heard 
about  a  conflict  resolution  style  of  management  that  reduced  the  level  of  tension  by  helping  people 
work  out  differences  themselves  and  seeking  reasonable  compromises. 

The  only  disappointment  we  heard  was  that  a  few  broad-band  researchers  were  frustrated  in 
attempts  to  gain  greater  acceptance  for  their  work  within  the  Air  Force.  They  felt  that  the  Division 
did  not  have  the  linkages  needed  to  go  the  next  step,  to  gain  a  better  hearing  for  the  efforts.  They 
were  also  frustrated  that  the  Division  did  not  have  a  means  of  overseeing  implementation  once  a 
methodology  left  Armstrong  Laboratory,  to  make  sure  that  the  application  was  successful.  By  not 
having  a  means  of  tracking  or  ensuring  follow-through,  maybe  the  impact  of  the  Division  on  the 
Air  Force  is  being  limited. 
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Overall,  the  success  of  management  at  the  Human  Engineering  Division  has  been  to  create  an 
environment  where  people  can  emerge  as  project  champions,  identifying  opportunities,  creating 
programs,  forging  alliances.  These  individuals  cannot  be  selected  or  identified  in  advance,  and 
rather  than  trying  for  a  top-down  direction,  the  management  tries  to  make  it  as  easy  as  possible  for 
the  high-impact  researchers  to  step  forward.  For  example,  the  division  tries  to  shoulder  as  much 
administrative  burden  as  possible  rather  than  passing  it  down  to  the  researchers.  And  while  the 
larger  changes  in  the  Department  of  Defense  have  required  a  greater  level  of  project  justification, 
and  stronger  paper  trails,  management  has  tried  to  keep  these  within  bounds  rather  than  using  them 
as  excuses  to  micromanage  the  work.  Instead,  upper  management  seeks  to  find  linkages  between 
projects  and  recognize  commonalities  that  can  be  exploited. 

Recommendations 


Synthesizing  these  case  studies,  we  can  identify  some  suggestions  to  help  the  Human  Engineering 
Division  replicate  the  successes  described  in  this  report. 

Expanding  the  tool  kit.  One  theme  in  these  case  studies  was  the  realization  that  a  new  type  of 
simulation  capability  or  measurement  technique  or  other  technology  could  be  brought  to  bear  on  a 
problem.  These  tools  were  discovered  at  professional  meetings,  visits  to  other  laboratories,  word- 
of-mouth  through  networks.  By  expanding  the  participation  in  these  networks,  it  may  be  possible 
to  strengthen  the  problem  finding  process.  The  recommendation  is  to  support  activities  that  will 
enable  researchers  to  learn  about  new  techniques. 

Expanding  the  communities.  By  helping  researchers  have  more  of  a  chance  to  interact  with 
users,  and  to  observe  operational  settings,  it  may  be  possible  to  strengthen  the  problem  finding 
process.  The  recommendation  is  to  support  activities  that  will  help  researchers  interact  with 
operational  communities. 

Transitioning  technology.  It  is  one  thing  to  state  this  as  a  Division  goal,  and  another  to  make 
it  happen.  While  official  programs  may  be  a  partial  solution,  more  likely  it  will  take  informal 
networks  at  the  level  of  upper  management  to  build  the  bridges  and  mechanisms  needed  to  turn 
research  projects  into  Air  Force  applications.  The  recommendation  is  for  management  to  become 
more  active  in  working  with  the  leadership  in  operational  communities. 

Reducing  the  contract  management  burdens.  Contract  management  and  administration  are 
a  part  of  the  research  scene,  and  there  may  be  little  the  Division  can  do  to  help,  but  it  clearly  does 
affect  the  process  and  forces  researchers  to  waste  time  finding  ways  to  get  things  done  without 
having  to  go  through  procurement  to  award  and  manage  contracts.  The  recommendation  is  for 
management  to  continue  its  efforts  to  protect  researchers  from  excessive  administrative  burdens. 

Dissemination.  The  case  studies  featured  too  many  examples  of  accidental  connections  to  be 
considered  coincidences.  The  challenge  is  to  increase  the  rate  of  these  happy  accidents. 
Professional  publications  and  technical  reports  may  not  be  read  by  the  user  communities  and  by 
collateral  communities  even  at  WPAFB.  Possibly  other  means,  such  as  informal  newsletters  or 
electronic  bulletins  describing  work  in  progress  may  be  helpful.  The  recommendation  is  to  develop 
better  strategies  for  describing  the  work  of  the  division  to  user  communities. 

Emissaries.  A  number  of  military  personnel  spend  a  short  time  in  the  Division,  often  because  of 
their  own  interest  in  human  factors,  and  then  move  to  other  positions  in  the  Air  Force.  Several  of 
the  case  studies  were  initiated  by  these  emissaries.  If  there  was  a  way  to  stay  in  touch  with  these 


57 


people,  and  keep  them  informed  about  the  work  of  the  Division,  it  might  increase  the  number  of 
happy  accidents.  The  recommendation  is  to  make  better  use  of  military  personnel  after  they 
complete  their  assignments  in  the  division. 

Internal  project  funds.  Based  on  this  small  sample,  it  appears  as  if  the  Division  is  doing  a 
good  job  with  its  internal  funding.  If  internal  funds  were  more  readily  available,  it  might 
discourage  the  user-centered  attitude  of  trying  to  find  problems  that  would  appeal  to  outside 
sponsors.  The  recommendation  is  to  maintain  the  current  strategy  of  disbursing  research  funds. 

Strategic  vision.  The  recommendation  is  to  continue  the  strategy  of  building  on  division 
strengths.  The  division  is  not  attempting  to  impose  a  purely  top-down  agenda,  which  rnight 
actually  interfere  with  the  problem  finding  activities  of  the  researchers.  Instead,  the  division  is 
taking  a  proactive  stance  of  taking  advantage  of  its  capabilities,  including  the  strengths  of  its 
leading  researchers,  to  achieve  greater  impact. 

In  summary,  the  case  studies  describe  a  set  of  successful  projects  that  have  benefited  the  Air  Force 
in  a  number  of  different  ways.  The  Armstrong  Laboratory's  reputation  is  built  on  the  work  of  its 
champions,  the  researchers  who  initiate  innovative  research.  The  work  conducted  at  the  lab  is  a 
balance  of  reactive  firefighting  and  anticipatory  fire  prevention.  The  researchers  choose  to  pursue 
different  modes  at  different  times.  From  all  the  possible  choices  of  projects  that  they  might  pursue, 
they  manage  to  pick  a  high  proportion  of  projects  which  often  have  an  immeasurable  impact  on  the 
human  engineering  of  Air  Force  systems. 

In  large  part,  the  projects  succeeded  because  of  the  initiative  of  the  researchers  themselves,  and 
their  ability  to  find  the  right  problems  to  work  on.  The  Human  Engineering  Division  has 
historically  been  able  to  create  an  environment  where  individual  initiative  is  encouraged.  These  case 
studies  are  testimony  to  the  merits  of  that  approach. 
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